{"record":{"id":"123e616e0cd1f5df","repo":"hashicorp/nomad","slug":"csi-plugins-lookup-failed-v","errorCode":null,"errorMessage":"csi_plugins lookup failed: %v","messagePattern":"csi_plugins lookup failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":1475,"sourceCode":"\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tinUseController[info.PluginID] = struct{}{}\n\t}\n\n\tfor _, info := range node.CSINodePlugins {\n\t\terr := upsertFn(info)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tinUseNode[info.PluginID] = struct{}{}\n\t}\n\n\t// remove the client node from any plugin that's not\n\t// running on it.\n\titer, err := txn.Get(TableCSIPlugins, \"id\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"csi_plugins lookup failed: %v\", err)\n\t}\n\tfor {\n\t\traw := iter.Next()\n\t\tif raw == nil {\n\t\t\tbreak\n\t\t}\n\t\tplug, ok := raw.(*structs.CSIPlugin)\n\t\tif !ok {\n\t\t\tcontinue\n\t\t}\n\t\tplug = plug.Copy()\n\n\t\tvar hadDelete bool\n\t\tif _, ok := inUseController[plug.ID]; !ok {\n\t\t\tif _, asController := plug.Controllers[node.ID]; asController {\n\t\t\t\terr := plug.DeleteNodeForType(node.ID, structs.CSIPluginTypeController)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err","sourceCodeStart":1457,"sourceCodeEnd":1493,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L1457-L1493","documentation":"Wraps a memdb iterator error when scanning the full csi_plugins table during node upsert to detach the client node from plugins no longer running on it. txn.Get(TableCSIPlugins, \"id\") failed, aborting the node registration transaction.","triggerScenarios":"upsertNodeTxn processing a node with CSIInfo; the table iterator acquisition fails (transaction or index state invalid), producing 'csi_plugins lookup failed'.","commonSituations":"Rare; seen alongside other memdb errors in server logs during state corruption or Nomad bug conditions, typically during node re-registration with CSI plugins attached.","solutions":["Retry the node registration; transient iterator failures usually clear on the next attempt.","Look at the wrapped %v error for the true memdb cause and fix that root cause.","Restart the Nomad server agent if the state store is persistently inconsistent.","Upgrade to the latest Nomad patch release for CSI/state-store fixes."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err != nil && strings.Contains(err.Error(), \"csi_plugins lookup failed\") {\n    logger.Error(\"state store iteration failed during node upsert\", \"err\", err)\n    return backoffRetry(registerNode)\n}","preventionTips":["Restart servers promptly if memdb errors repeat — they signal state corruption.","Keep Nomad upgraded to the latest patch release.","Watch node registration success rates in telemetry.","Avoid mixing heavily mismatched client/server versions in one cluster."],"tags":["nomad","csi","state-store","memdb"],"backgroundTag":"csi-plugin-registration-failure","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}