{"record":{"id":"54352c05064da8dd","repo":"netbirdio/netbird","slug":"remove-prerouting-rule-w","errorCode":null,"errorMessage":"remove prerouting rule: %w","messagePattern":"remove prerouting rule: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/firewall/nftables/router_linux.go","lineNumber":787,"sourceCode":"\t}\n\n\texprs = append(exprs,\n\t\t&expr.Immediate{\n\t\t\tRegister: 1,\n\t\t\tData:     binaryutil.NativeEndian.PutUint32(markValue),\n\t\t},\n\t\t&expr.Meta{\n\t\t\tKey:            expr.MetaKeyMARK,\n\t\t\tSourceRegister: true,\n\t\t\tRegister:       1,\n\t\t},\n\t)\n\n\truleKey := firewall.GenKey(firewall.PreroutingFormat, pair)\n\n\tif _, exists := r.rules[ruleKey]; exists {\n\t\tif err := r.removeNatRule(pair); err != nil {\n\t\t\treturn fmt.Errorf(\"remove prerouting rule: %w\", err)\n\t\t}\n\t}\n\n\t// Ensure nat rules come first, so the mark can be overwritten.\n\t// Currently overwritten by the dst-type LOCAL rules for redirected traffic.\n\tr.rules[ruleKey] = r.conn.InsertRule(&nftables.Rule{\n\t\tTable:    r.workTable,\n\t\tChain:    r.chains[chainNameManglePrerouting],\n\t\tExprs:    exprs,\n\t\tUserData: []byte(ruleKey),\n\t})\n\n\treturn nil\n}\n\n// addPostroutingRules adds the masquerade rules\nfunc (r *router) addPostroutingRules() {\n\t// First masquerade rule for traffic coming in from WireGuard interface","sourceCodeStart":769,"sourceCodeEnd":805,"githubUrl":"https://github.com/netbirdio/netbird/blob/93e97f4bf1ad715072dcb3fb6cdb1763431b5a9c/client/firewall/nftables/router_linux.go#L769-L805","documentation":"Returned by addNatRule (router_linux.go:786) when the rule key firewall.GenKey(PreroutingFormat, pair) already exists in r.rules and removeNatRule fails while replacing it. removeNatRule (line 1495) deletes the stored rule via conn.DelRule, drops the map entry, and decrements the set counter; it fails on a zero-handle rule queued but never flushed (DelRule marshal error), on a deleteIpSet flush failure inside the decrement, or on ENOENT-class kernel errors at its callers' flush. NetBird re-inserts prerouting rules on every route update specifically to keep them first in the chain (comment lines 791-792), so this fires on idempotent re-adds.","triggerScenarios":"Re-applying a masqueraded route (network-map refresh) after a previous AddNatRule flush failed, leaving a Handle==0 rule in r.rules; or the set referenced by the old rule failing to delete during the decrement (EBUSY/ENOENT).","commonSituations":"Frequent route updates on peers with unstable nftables access; firewalld deleting the table between updates, desynchronizing handles; agents that previously hit error 693 and kept stale entries.","solutions":["Call refreshRulesMap before replacing (AddNatRule already does, line 683) so the existing key maps to a kernel-fresh handle; if refresh shows the rule gone, the key disappears and this branch is skipped.","Handle Handle==0 in removeNatRule as a stale entry (delete from map, decrement, return nil) — mirroring lines 1504-1511 which already do this.","Fix the underlying decrement failure using error 687 guidance.","Restart the agent to rebuild r.rules from the kernel when stale entries persist."],"exampleFix":"// before\nif _, exists := r.rules[ruleKey]; exists {\n    if err := r.removeNatRule(pair); err != nil {\n        return fmt.Errorf(\"remove prerouting rule: %w\", err)\n    }\n}\n\n// after\nif _, exists := r.rules[ruleKey]; exists {\n    if err := r.removeNatRule(pair); err != nil {\n        if isErrno(err, unix.ENOENT) || strings.Contains(err.Error(), \"handle\") {\n            // stale local entry; refresh resyncs and the insert below replaces it\n            log.Warnf(\"stale prerouting rule %s: %v\", ruleKey, err)\n            delete(r.rules, ruleKey)\n        } else {\n            return fmt.Errorf(\"remove prerouting rule: %w\", err)\n        }\n    }\n}","handlingStrategy":"retry","validationCode":"// Ensure the existing rule has a live handle before attempting replacement\nexisting, ok := r.rules[ruleKey]\nif ok && existing.Handle == 0 {\n    _ = r.refreshRulesMap() // resync handles from kernel\n    existing, ok = r.rules[ruleKey]\n    if ok && existing.Handle == 0 {\n        delete(r.rules, ruleKey) // truly stale; let the insert below replace it\n    }\n}","typeGuard":"func hasLiveHandle(rule *nftables.Rule) bool {\n\treturn rule != nil && rule.Handle != 0 && rule.Table != nil && rule.Chain != nil\n}","tryCatchPattern":"if err := r.removeNatRule(pair); err != nil {\n    if isErrno(err, unix.ENOENT, unix.EBUSY) {\n        delete(r.rules, ruleKey) // stale or set-in-use; insert below still replaces the rule\n        log.Warnf(\"prerouting replace for %s: %v\", ruleKey, err)\n    } else {\n        return fmt.Errorf(\"remove prerouting rule: %w\", err)\n    }\n}","preventionTips":["Refresh the rule map immediately before rule replacement so existing keys carry kernel handles.","Treat ENOENT during replacement as safe: the rule is gone, the new insert recreates it.","Keep prerouting rule keys stable (GenKey with the same pair) so re-adds replace rather than accumulate.","After any flush failure, purge handle-less entries from r.rules so later replacements cannot trip on them."],"tags":["go","nftables","nat","stale-state","routing","firewall","linux"],"backgroundTag":null,"analyzedSha":"93e97f4bf1ad715072dcb3fb6cdb1763431b5a9c","analyzedAt":"2026-08-16T03:09:19.136Z","schemaVersion":2},"datasetVersion":"2026-08-16T08:17:34.114Z"}