netbirdio/netbird · error

remove prerouting rule: %w

Error message

remove prerouting rule: %w

What it means

Returned by addNatRule (router_linux.go:786) when the rule key firewall.GenKey(PreroutingFormat, pair) already exists in r.rules and removeNatRule fails while replacing it. removeNatRule (line 1495) deletes the stored rule via conn.DelRule, drops the map entry, and decrements the set counter; it fails on a zero-handle rule queued but never flushed (DelRule marshal error), on a deleteIpSet flush failure inside the decrement, or on ENOENT-class kernel errors at its callers' flush. NetBird re-inserts prerouting rules on every route update specifically to keep them first in the chain (comment lines 791-792), so this fires on idempotent re-adds.

Source

Thrown at client/firewall/nftables/router_linux.go:787

	}

	exprs = append(exprs,
		&expr.Immediate{
			Register: 1,
			Data:     binaryutil.NativeEndian.PutUint32(markValue),
		},
		&expr.Meta{
			Key:            expr.MetaKeyMARK,
			SourceRegister: true,
			Register:       1,
		},
	)

	ruleKey := firewall.GenKey(firewall.PreroutingFormat, pair)

	if _, exists := r.rules[ruleKey]; exists {
		if err := r.removeNatRule(pair); err != nil {
			return fmt.Errorf("remove prerouting rule: %w", err)
		}
	}

	// Ensure nat rules come first, so the mark can be overwritten.
	// Currently overwritten by the dst-type LOCAL rules for redirected traffic.
	r.rules[ruleKey] = r.conn.InsertRule(&nftables.Rule{
		Table:    r.workTable,
		Chain:    r.chains[chainNameManglePrerouting],
		Exprs:    exprs,
		UserData: []byte(ruleKey),
	})

	return nil
}

// addPostroutingRules adds the masquerade rules
func (r *router) addPostroutingRules() {
	// First masquerade rule for traffic coming in from WireGuard interface

View on GitHub (pinned to 93e97f4bf1)

Solutions

1. Call refreshRulesMap before replacing (AddNatRule already does, line 683) so the existing key maps to a kernel-fresh handle; if refresh shows the rule gone, the key disappears and this branch is skipped.
2. Handle Handle==0 in removeNatRule as a stale entry (delete from map, decrement, return nil) — mirroring lines 1504-1511 which already do this.
3. Fix the underlying decrement failure using error 687 guidance.
4. Restart the agent to rebuild r.rules from the kernel when stale entries persist.

Example fix

// before
if _, exists := r.rules[ruleKey]; exists {
    if err := r.removeNatRule(pair); err != nil {
        return fmt.Errorf("remove prerouting rule: %w", err)
    }
}

// after
if _, exists := r.rules[ruleKey]; exists {
    if err := r.removeNatRule(pair); err != nil {
        if isErrno(err, unix.ENOENT) || strings.Contains(err.Error(), "handle") {
            // stale local entry; refresh resyncs and the insert below replaces it
            log.Warnf("stale prerouting rule %s: %v", ruleKey, err)
            delete(r.rules, ruleKey)
        } else {
            return fmt.Errorf("remove prerouting rule: %w", err)
        }
    }
}

// Ensure the existing rule has a live handle before attempting replacement
existing, ok := r.rules[ruleKey]
if ok && existing.Handle == 0 {
    _ = r.refreshRulesMap() // resync handles from kernel
    existing, ok = r.rules[ruleKey]
    if ok && existing.Handle == 0 {
        delete(r.rules, ruleKey) // truly stale; let the insert below replace it
    }
}

func hasLiveHandle(rule *nftables.Rule) bool {
	return rule != nil && rule.Handle != 0 && rule.Table != nil && rule.Chain != nil
}

if err := r.removeNatRule(pair); err != nil {
    if isErrno(err, unix.ENOENT, unix.EBUSY) {
        delete(r.rules, ruleKey) // stale or set-in-use; insert below still replaces the rule
        log.Warnf("prerouting replace for %s: %v", ruleKey, err)
    } else {
        return fmt.Errorf("remove prerouting rule: %w", err)
    }
}

When it happens

Trigger: Re-applying a masqueraded route (network-map refresh) after a previous AddNatRule flush failed, leaving a Handle==0 rule in r.rules; or the set referenced by the old rule failing to delete during the decrement (EBUSY/ENOENT).

Common situations: Frequent route updates on peers with unstable nftables access; firewalld deleting the table between updates, desynchronizing handles; agents that previously hit error 693 and kept stale entries.

Related errors

• add inverse nat rule: %w
• remove legacy routing rule: %w
• add nat rule: %w
• insert rules for %s: %w
• decrement set counter: %w

AI-assisted analysis of netbirdio/netbird@93e97f4bf1 (2026-08-16). Data as JSON: /api/errors/54352c05064da8dd. Report an issue: GitHub.