{"record":{"id":"38f816542709e18b","repo":"hashicorp/nomad","slug":"errcnicheckfailed","errorCode":"ErrCNICheckFailed","errorMessage":"%w: %w","messagePattern":"%w: %w","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/networking_cni.go","lineNumber":199,"sourceCode":"\t\tcniArgs[ConsulIPTablesConfigEnvVar] = string(iptablesCfg)\n\t}\n\n\tif !created {\n\t\t// The netns will not be created if it already exists, typically on\n\t\t// agent restart. If the configuration of a prexisting netns is wrong\n\t\t// (ex. after a host reboot for docker created netns), networking will\n\t\t// be broken. CNI's ADD command is not idempotent so we can't simply try\n\t\t// again. Run CHECK to verify the network is still valid. Older plugins\n\t\t// have a broken CHECK, so we have to allow the buggy behavior in the\n\t\t// case of a host reboot with docker-created netns there.\n\t\tcniVersion, err := version.NewSemver(c.nodeAttrs[\"plugins.cni.version.bridge\"])\n\t\tif err == nil && supportsCNICheck.Check(cniVersion) {\n\t\t\terr := c.cni.Check(ctx, alloc.ID, spec.Path,\n\t\t\t\tc.nsOpts.withCapabilityPortMap(portMaps.ports),\n\t\t\t\tc.nsOpts.withArgs(cniArgs),\n\t\t\t)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"%w: %w\", ErrCNICheckFailed, err)\n\t\t\t}\n\t\t} else {\n\t\t\tc.logger.Debug(\"network namespace exists but could not check if networking is valid because bridge plugin version was <1.3.0: continuing anyways\")\n\t\t\treturn nil, nil\n\t\t}\n\t\tc.logger.Trace(\"network namespace exists and passed check: skipping setup\")\n\t\treturn nil, nil\n\t}\n\n\t// Depending on the version of bridge cni plugin used, a known race could occure\n\t// where two alloc attempt to create the nomad bridge at the same time, resulting\n\t// in one of them to fail. This rety attempts to overcome those erroneous failures.\n\tconst retry = 3\n\tvar firstError error\n\tvar res *cni.Result\n\tfor attempt := 1; ; attempt++ {\n\t\tvar err error\n\t\tif res, err = c.cni.Setup(ctx, alloc.ID, spec.Path,","sourceCodeStart":181,"sourceCodeEnd":217,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/networking_cni.go#L181-L217","documentation":"When an allocation's network namespace already exists, Setup runs CNI CHECK to validate existing networking (requires bridge plugin >=1.3.0). If cni.Check fails, Nomad wraps it with the sentinel ErrCNICheckFailed so callers can detect a broken/missing netns setup distinctly from a fresh-setup failure.","triggerScenarios":"Alloc netns exists, the CNI config reports support for CHECK (bridge plugin version >=1.3.0), and c.cni.Check(...) returns an error — e.g. netns missing/corrupt, interface gone, iptables chain missing, or plugin binary changed since ADD.","commonSituations":"Client restart with stale netns after host reboot wiped iptables/links; operator deleted the bridge or veth manually; CNI plugin upgrade changing behavior; disk/netns cleanup utilities removing /var/run/netns entries.","solutions":["Let Nomad tear down and re-create: stop the allocation (nomad alloc stop) or restart the client so a fresh CNI ADD runs.","Inspect the wrapped error from cni.Check for the failing plugin and fix host networking state (recreate bridge, restore iptables).","Match CNI plugin versions on the host with the version reported by the CNI conf (>=1.3.0 required for CHECK).","Handle ErrCNICheckFailed in custom code with errors.Is to distinguish from other setup failures."],"exampleFix":"// before\nif err := c.cni.Check(...); err != nil { return nil, err }\n// after\nif err := c.cni.Check(...); err != nil {\n  if errors.Is(err, ErrCNICheckFailed) {\n    // recreate netns via full teardown/setup\n  }\n  return nil, fmt.Errorf(\"%w: %w\", ErrCNICheckFailed, err)\n}","handlingStrategy":"try-catch","validationCode":"// detect stale netns before setup\nif _, err := os.Stat(spec.Path); err != nil { /* netns missing; force fresh ADD */ }","typeGuard":"func isCNICheckFailed(err error) bool {\n  return errors.Is(err, ErrCNICheckFailed)\n}","tryCatchPattern":"status, err := cfg.Setup(ctx, alloc, spec, created)\nif isCNICheckFailed(err) {\n  // teardown netns and retry Setup once for a clean CNI ADD\n  _ = cfg.Teardown(ctx, alloc, spec, false)\n  status, err = cfg.Setup(ctx, alloc, spec, false)\n}","preventionTips":["Pin CNI bridge plugin >=1.3.0 on hosts where allocs resume across restarts.","Avoid manual deletion of nomad bridges/veths on live clients.","Nomad alloc stop + restart to recover rather than hand-editing netns state."],"tags":["nomad","cni","netns-check","sentinel-error"],"backgroundTag":"cni-check-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}