{"record":{"id":"02a485cbff3503d0","repo":"netbirdio/netbird","slug":"remove-legacy-routing-rule-w-02a485","errorCode":null,"errorMessage":"remove legacy routing rule: %w","messagePattern":"remove legacy routing rule: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/firewall/nftables/router_linux.go","lineNumber":985,"sourceCode":"// addLegacyRouteRule adds a legacy routing rule for mgmt servers pre route acls\nfunc (r *router) addLegacyRouteRule(pair firewall.RouterPair) error {\n\tsourceExp, err := r.applyNetwork(pair.Source, nil, true)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"apply source: %w\", err)\n\t}\n\n\tdestExp, err := r.applyNetwork(pair.Destination, nil, false)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"apply destination: %w\", err)\n\t}\n\n\texprs := buildLegacyRouteRuleExpressions(sourceExp, destExp)\n\n\truleKey := firewall.GenKey(firewall.ForwardingFormat, pair)\n\n\tif _, exists := r.rules[ruleKey]; exists {\n\t\tif err := r.removeLegacyRouteRule(pair); err != nil {\n\t\t\treturn fmt.Errorf(\"remove legacy routing rule: %w\", err)\n\t\t}\n\t}\n\n\tr.rules[ruleKey] = r.conn.AddRule(&nftables.Rule{\n\t\tTable:    r.workTable,\n\t\tChain:    r.chains[chainNameRoutingFw],\n\t\tExprs:    exprs,\n\t\tUserData: []byte(ruleKey),\n\t})\n\treturn nil\n}\n\n// removeLegacyRouteRule removes a legacy routing rule for mgmt servers pre route acls\nfunc (r *router) removeLegacyRouteRule(pair firewall.RouterPair) error {\n\truleKey := firewall.GenKey(firewall.ForwardingFormat, pair)\n\n\trule, exists := r.rules[ruleKey]\n\tif !exists {","sourceCodeStart":967,"sourceCodeEnd":1003,"githubUrl":"https://github.com/netbirdio/netbird/blob/93e97f4bf1ad715072dcb3fb6cdb1763431b5a9c/client/firewall/nftables/router_linux.go#L967-L1003","documentation":"Returned by addLegacyRouteRule (router_linux.go:984) when the forwarding rule key firewall.GenKey(ForwardingFormat, pair) already exists and removeLegacyRouteRule fails. removeLegacyRouteRule (line 999) handles a Handle==0 stale entry gracefully (lines 1007-1014) but errors on DelRule marshal/kernel failures (\"remove legacy forwarding rule ...\", line 1017) or on a decrementSetCounter failure when the last set reference drops (line 1025 -> deleteIpSet flush, error 686). Legacy rules are re-added on every route update, making this replacement path hot whenever legacy mode is active.","triggerScenarios":"Re-applying a legacy route while the old rule's set fails to delete (EBUSY: still referenced; ENOENT: already gone; EPERM), or after a previous flush failure left a rule whose expressions cannot be marshaled by DelRule despite having a handle.","commonSituations":"Long-lived legacy-mode peers accumulating leaked sets so every decrement risks EBUSY; firewalld interference desynchronizing handles; RemoveNatRule hitting the same removeLegacyRouteRule (line 1482) during teardown.","solutions":["Run refreshRulesMap before replacement so the forwarding key maps to a live handle (AddNatRule does this at line 683 — ensure the error from it was not ignored).","For EBUSY decrements, let the next removal cycle retry: counter entries persist and self-heal.","For ENOENT, treat as success — the rule is already gone from the kernel.","Restart the agent to rebuild map, counters, and work table when stale state keeps blocking replacement."],"exampleFix":"// before\nif _, exists := r.rules[ruleKey]; exists {\n    if err := r.removeLegacyRouteRule(pair); err != nil {\n        return fmt.Errorf(\"remove legacy routing rule: %w\", err)\n    }\n}\n\n// after\nif _, exists := r.rules[ruleKey]; exists {\n    if err := r.removeLegacyRouteRule(pair); err != nil {\n        if isErrno(err, unix.ENOENT, unix.EBUSY) {\n            // kernel rule gone or set still referenced; insert below still replaces it\n            log.Warnf(\"replace legacy rule %s: %v\", ruleKey, err)\n            delete(r.rules, ruleKey)\n        } else {\n            return fmt.Errorf(\"remove legacy routing rule: %w\", err)\n        }\n    }\n}","handlingStrategy":"retry","validationCode":"// Ensure the forwarding rule has a live handle before replacement\nexisting, ok := r.rules[ruleKey]\nif ok && existing.Handle == 0 {\n    _ = r.refreshRulesMap()\n    if existing, ok = r.rules[ruleKey]; ok && existing.Handle == 0 {\n        delete(r.rules, ruleKey) // stale; insert below recreates it\n    }\n}","typeGuard":"func hasLiveHandle(rule *nftables.Rule) bool {\n\treturn rule != nil && rule.Handle != 0 && rule.Table != nil && rule.Chain != nil\n}\n\nfunc isReplaceTolerable(err error) bool {\n\treturn isErrno(err, unix.ENOENT, unix.EBUSY)\n}","tryCatchPattern":"if err := r.removeLegacyRouteRule(pair); err != nil {\n    if isReplaceTolerable(err) {\n        // rule gone (ENOENT) or set still referenced (EBUSY): insert below replaces anyway\n        log.Warnf(\"legacy replace %s: %v\", ruleKey, err)\n        delete(r.rules, ruleKey)\n    } else {\n        return fmt.Errorf(\"remove legacy routing rule: %w\", err)\n    }\n}","preventionTips":["Refresh the rule map before legacy rule replacement so handles are live.","Accept ENOENT/EBUSY on replacement as tolerable — the subsequent AddRule recreates the rule regardless.","On teardown, RemoveNatRule also calls removeLegacyRouteRule; keep its multierror handling non-fatal for the same reasons.","Restart the agent to rebuild counters when legacy replacement failures persist across refreshes."],"tags":["go","nftables","legacy","stale-state","routing","firewall","linux"],"backgroundTag":null,"analyzedSha":"93e97f4bf1ad715072dcb3fb6cdb1763431b5a9c","analyzedAt":"2026-08-16T03:09:19.136Z","schemaVersion":2},"datasetVersion":"2026-08-16T08:17:34.114Z"}