{"record":{"id":"09868031e4da182e","repo":"hashicorp/nomad","slug":"plugin-missing-controller-s","errorCode":null,"errorMessage":"plugin missing controller: %s","messagePattern":"plugin missing controller: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/structs/csi.go","lineNumber":1339,"sourceCode":"\t\t\treturn c.NodeInfo.SupportsExpand\n\t\tcase CSINodeSupportsCondition:\n\t\t\treturn c.NodeInfo.SupportsCondition\n\t\tdefault:\n\t\t\treturn false\n\t\t}\n\t}\n\treturn false\n}\n\n// AddPlugin adds a single plugin running on the node. Called from state.NodeUpdate in a\n// transaction\nfunc (p *CSIPlugin) AddPlugin(nodeID string, info *CSIInfo) error {\n\tif info.ControllerInfo != nil {\n\t\tp.ControllerRequired = info.RequiresControllerPlugin\n\t\tprev, ok := p.Controllers[nodeID]\n\t\tif ok {\n\t\t\tif prev == nil {\n\t\t\t\treturn fmt.Errorf(\"plugin missing controller: %s\", nodeID)\n\t\t\t}\n\t\t\tif prev.Healthy {\n\t\t\t\tp.ControllersHealthy -= 1\n\t\t\t}\n\t\t}\n\n\t\t// note: for this to work as expected, only a single\n\t\t// controller for a given plugin can be on a given Nomad\n\t\t// client, they also conflict on the client so this should be\n\t\t// ok\n\t\tif prev != nil || info.Healthy {\n\t\t\tp.Controllers[nodeID] = info\n\t\t}\n\t\tif info.Healthy {\n\t\t\tp.ControllersHealthy += 1\n\t\t}\n\t}\n","sourceCodeStart":1321,"sourceCodeEnd":1357,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/structs/csi.go#L1321-L1357","documentation":"CSIPlugin.AddPlugin returns this when it is about to update the controller entry for nodeID and finds the map says a controller exists ('ok' is true) but the stored pointer is nil. This internally inconsistent state (map key present, value nil) prevents a safe health decrement, so it surfaces an error instead of panicking.","triggerScenarios":"A CSI plugin fingerprint update (Node.UpdateCSIPlugin RPC) for a node whose Controllers map contains a nil-valued entry for that nodeID while info.ControllerInfo != nil — e.g. after partial de-registration, a nil overwriting bug, or an incomplete state restore.","commonSituations":"A plugin transitions from controller-present to absent and back while the fingerprint loop fires concurrently; plugin data deserialized from raft where the controller was tombstoned to nil; region/agent restart with partially flushed CSI plugin state.","solutions":["Restart the Nomad client agent so it re-fingerprints the plugin and repopulates Controllers with a valid CSIInfo.","Run nomad plugin status <plugin-id>; if the plugin is half-registered, deregister and let the client re-register (delete the plugin or stop/start the plugin task).","If the client keeps failing, remove the stale client node (nomad node eligibility -disable / stop client) so the plugin entry is rebuilt.","Report/pin down the writer that set the nil controller (check nomad agent logs around 'csi') — it is a state-consistency bug, not a user configuration error."],"exampleFix":"// before\n// map entry tombstoned to nil by earlier teardown\np.Controllers[nodeID] = nil\n// after\n// delete the key instead of storing nil so the map never claims a missing controller\ndelete(p.Controllers, nodeID)\np.ControllersHealthy = 0","handlingStrategy":"try-catch","validationCode":"prev, ok := p.Controllers[nodeID]\nif ok && prev == nil {\n    // repair instead of erroring: drop the nil tombstone\n    delete(p.Controllers, nodeID)\n    prev, ok = nil, false\n}\n_ = prev\n_ = ok","typeGuard":"func hasController(p *structs.CSIPlugin, nodeID string) bool {\n    c, ok := p.Controllers[nodeID]\n    return ok && c != nil\n}","tryCatchPattern":"err := plugin.AddPlugin(nodeID, info)\nif err != nil {\n    if strings.Contains(err.Error(), \"plugin missing controller\") {\n        // state inconsistency: restart client or deregister/re-register the plugin\n        return reconcilePlugin(pluginID)\n    }\n    return err\n}","preventionTips":["Delete map keys instead of assigning nil when tearing down controller entries.","Serialize plugin updates per node to avoid teardown racing fingerprint updates.","After client restarts, run nomad plugin status to confirm controllers are healthy.","Upgrade Nomad if you see this repeatedly — it indicates a known class of state bug."],"tags":["nomad","csi","plugin","state-consistency"],"backgroundTag":"csi-plugin-state-corruption","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}