{"record":{"id":"cadf486f742b8640","repo":"hashicorp/nomad","slug":"plugin-missing-node-s","errorCode":null,"errorMessage":"plugin missing node: %s","messagePattern":"plugin missing node: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/structs/csi.go","lineNumber":1362,"sourceCode":"\t\t}\n\n\t\t// note: for this to work as expected, only a single\n\t\t// controller for a given plugin can be on a given Nomad\n\t\t// client, they also conflict on the client so this should be\n\t\t// ok\n\t\tif prev != nil || info.Healthy {\n\t\t\tp.Controllers[nodeID] = info\n\t\t}\n\t\tif info.Healthy {\n\t\t\tp.ControllersHealthy += 1\n\t\t}\n\t}\n\n\tif info.NodeInfo != nil {\n\t\tprev, ok := p.Nodes[nodeID]\n\t\tif ok {\n\t\t\tif prev == nil {\n\t\t\t\treturn fmt.Errorf(\"plugin missing node: %s\", nodeID)\n\t\t\t}\n\t\t\tif prev.Healthy {\n\t\t\t\tp.NodesHealthy -= 1\n\t\t\t}\n\t\t}\n\t\tif prev != nil || info.Healthy {\n\t\t\tp.Nodes[nodeID] = info\n\t\t}\n\t\tif info.Healthy {\n\t\t\tp.NodesHealthy += 1\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// DeleteNode removes all plugins from the node. Called from state.DeleteNode in a\n// transaction","sourceCodeStart":1344,"sourceCodeEnd":1380,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/structs/csi.go#L1344-L1380","documentation":"The node-registry mirror of the controller error: AddPlugin detects the Nodes map contains a nil CSIInfo for nodeID while an update for info.NodeInfo arrives. Since the previous entry is unusable, the function refuses to decrement NodesHealthy and returns this error rather than dereferencing nil.","triggerScenarios":"A Node.UpdateCSIPlugin fingerprint update carrying NodeInfo for a node whose Nodes map slot exists but holds nil — typically from a prior deregistration that nulled the value, a concurrent fingerprint update, or corrupted plugin state in raft.","commonSituations":"CSI node plugin restarting rapidly so teardown and re-fingerprint interleave; snapshot restore where node entries were pruned incompletely; bug in an older Nomad version that set map values to nil instead of deleting keys.","solutions":["Restart the Nomad client agent to force a clean re-fingerprint of the node plugin.","Check nomad plugin status <id>; deregister the broken plugin and let the client re-register it (restart the CSI plugin task).","Upgrade Nomad if on an older release — nil-value inconsistencies in CSIPlugin maps were fixed by deleting keys instead of nulling them.","As a last resort, drain and remove the affected client node so its plugin entries are fully rebuilt."],"exampleFix":"// before\np.Nodes[nodeID] = nil // leaves key present with nil value\n// after\ndelete(p.Nodes, nodeID)\np.NodesHealthy = 0","handlingStrategy":"try-catch","validationCode":"prev, ok := p.Nodes[nodeID]\nif ok && prev == nil {\n    // nil tombstone: remove it before applying the new NodeInfo\n    delete(p.Nodes, nodeID)\n    prev, ok = nil, false\n}\n_ = prev\n_ = ok","typeGuard":"func hasNode(p *structs.CSIPlugin, nodeID string) bool {\n    n, ok := p.Nodes[nodeID]\n    return ok && n != nil\n}","tryCatchPattern":"err := plugin.AddPlugin(nodeID, info)\nif err != nil {\n    if strings.Contains(err.Error(), \"plugin missing node\") {\n        // client-side state is inconsistent; restart the client agent or\n        // restart the CSI node plugin task so it re-fingerprints\n        return restartCSIPluginTask(nodeID)\n    }\n    return err\n}","preventionTips":["Never store nil CSIInfo in the Nodes map; delete the key on teardown.","Avoid rapid plugin restarts that interleave deregistration and fingerprinting.","Verify node plugin health via nomad node status <node> -verbose (CSI drivers section).","If persistent, drain/remove the client so plugin entries are fully rebuilt."],"tags":["nomad","csi","plugin","node-registration","state-consistency"],"backgroundTag":"csi-plugin-state-corruption","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}