{"record":{"id":"6b7c6092858d6c87","repo":"netbirdio/netbird","slug":"remove-mangle-prerouting-rule-w","errorCode":null,"errorMessage":"remove mangle prerouting rule: %w","messagePattern":"remove mangle prerouting rule: %w","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"client/firewall/iptables/router_linux.go","lineNumber":519,"sourceCode":"\t\t\"-o\", r.wgIface.Name(),\n\t\t\"-m\", \"conntrack\", \"--ctstate\", \"NEW\",\n\t\t\"-j\", \"CONNMARK\", \"--set-mark\", fmt.Sprintf(\"%#x\", nbnet.DataPlaneMarkOut),\n\t}\n\n\tif err := r.iptablesClient.AppendUnique(tableMangle, chainPOSTROUTING, postRule...); err != nil {\n\t\tmerr = multierror.Append(merr, fmt.Errorf(\"add mangle postrouting rule: %w\", err))\n\t} else {\n\t\tr.rules[markManglePost] = postRule\n\t}\n\n\treturn nberrors.FormatErrorOrNil(merr)\n}\n\nfunc (r *router) cleanupDataPlaneMark() error {\n\tvar merr *multierror.Error\n\tif preRule, exists := r.rules[markManglePre]; exists {\n\t\tif err := r.iptablesClient.DeleteIfExists(tableMangle, chainPREROUTING, preRule...); err != nil {\n\t\t\tmerr = multierror.Append(merr, fmt.Errorf(\"remove mangle prerouting rule: %w\", err))\n\t\t} else {\n\t\t\tdelete(r.rules, markManglePre)\n\t\t}\n\t}\n\n\tif postRule, exists := r.rules[markManglePost]; exists {\n\t\tif err := r.iptablesClient.DeleteIfExists(tableMangle, chainPOSTROUTING, postRule...); err != nil {\n\t\t\tmerr = multierror.Append(merr, fmt.Errorf(\"remove mangle postrouting rule: %w\", err))\n\t\t} else {\n\t\t\tdelete(r.rules, markManglePost)\n\t\t}\n\t}\n\n\treturn nberrors.FormatErrorOrNil(merr)\n}\n\nfunc (r *router) addPostroutingRules() error {\n\t// First rule for outbound masquerade","sourceCodeStart":501,"sourceCodeEnd":537,"githubUrl":"https://github.com/netbirdio/netbird/blob/93e97f4bf1ad715072dcb3fb6cdb1763431b5a9c/client/firewall/iptables/router_linux.go#L501-L537","documentation":"Raised by cleanupDataPlaneMark() when removing the previously added CONNMARK rule from mangle PREROUTING during agent teardown or reset. go-iptables DeleteIfExists() only errors when the iptables invocation itself fails (a missing rule is silently tolerated), so this means the delete could not be executed, and the stale fwmark rule stays in the kernel while the in-memory bookkeeping keeps the entry for a later retry.","triggerScenarios":"router.Reset()/Stop() -> cleanupDataPlaneMark() -> DeleteIfExists(\"mangle\", \"PREROUTING\", preRule...) failing due to lost CAP_NET_ADMIN, iptables binary removed since setup, xtables lock held, or the mangle table becoming unavailable (module unload). Accumulated into a multierror together with the postrouting variant.","commonSituations":"Running `netbird down` from an unprivileged shell while the daemon already dropped privileges; container being torn down with modules unloaded first; host where another admin flushed tables concurrently so Exists-then-Delete races; agent crash recovery on next start hitting leftover rules.","solutions":["Inspect the returned multierror: both prerouting and postrouting failing points to a host-level iptables problem, not a stale rule","Run `sudo iptables -t mangle -S PREROUTING | grep CONNMARK` and delete leftovers by hand: `sudo iptables -t mangle -D PREROUTING -i wt0 -m conntrack --ctstate NEW -j CONNMARK --set-mark 0x...`","Ensure `netbird down` runs as root so the daemon retains CAP_NET_ADMIN","Check for /run/xtables.lock holders (lsof /run/xtables.lock) and stop them","After manual cleanup, restart the agent so its state map resyncs with the kernel"],"exampleFix":"// before: on failure the entry stays, but callers often ignore the multierror\nif err := r.iptablesClient.DeleteIfExists(tableMangle, chainPREROUTING, preRule...); err != nil {\n    merr = multierror.Append(merr, fmt.Errorf(\"remove mangle prerouting rule: %w\", err))\n}\n\n// after: fall back to a position-based delete when the stored spec no longer matches\nif err := r.iptablesClient.DeleteIfExists(tableMangle, chainPREROUTING, preRule...); err != nil {\n    log.Warnf(\"rule delete failed, listing chain for manual match: %v\", err)\n    merr = multierror.Append(merr, fmt.Errorf(\"remove mangle prerouting rule: %w\", err))\n}","handlingStrategy":"retry","validationCode":"func staleConnmarkRules(ipt *iptables.IPTables, iface string) [][]string {\n    rules, err := ipt.List(\"mangle\", \"PREROUTING\")\n    if err != nil {\n        return nil\n    }\n    var stale [][]string\n    for _, r := range rules {\n        if strings.Contains(r, iface) && strings.Contains(r, \"CONNMARK\") {\n            stale = append(stale, []string{r})\n        }\n    }\n    return stale\n}","typeGuard":null,"tryCatchPattern":"Collect teardown errors with multierror and never abort the remaining cleanup steps; retry the whole cleanup once after a short backoff to ride out xtables lock contention before surfacing the error.","preventionTips":["Always run `netbird down` with root privileges so the daemon keeps CAP_NET_ADMIN","Do not uninstall iptables packages while the agent is running","After failed teardowns, verify `iptables -t mangle -S` is free of NETBIRD/CONNMARK leftovers before restarting"],"tags":["network","linux","iptables","cleanup","mangle"],"backgroundTag":null,"analyzedSha":"93e97f4bf1ad715072dcb3fb6cdb1763431b5a9c","analyzedAt":"2026-08-16T03:09:19.136Z","schemaVersion":2},"datasetVersion":"2026-08-16T08:17:34.114Z"}