{"record":{"id":"1dc15df64d35525e","repo":"hashicorp/nomad","slug":"volume-lookup-failed-v","errorCode":null,"errorMessage":"volume lookup failed: %v","messagePattern":"volume lookup failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":2746,"sourceCode":"\n\tif obj == nil {\n\t\treturn nil, nil\n\t}\n\tvol := obj.(*structs.CSIVolume)\n\n\t// we return the volume with the plugins denormalized by default,\n\t// because the scheduler needs them for feasibility checking\n\treturn s.csiVolumeDenormalizePluginsTxn(txn, vol.Copy())\n}\n\n// CSIVolumesByPluginID looks up csi_volumes by pluginID. Caller should\n// snapshot if it wants to also denormalize the plugins.\nfunc (s *StateStore) CSIVolumesByPluginID(ws memdb.WatchSet, namespace, prefix, pluginID string) (memdb.ResultIterator, error) {\n\ttxn := s.db.ReadTxn()\n\n\titer, err := txn.Get(TableCSIVolumes, \"plugin_id\", pluginID)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"volume lookup failed: %v\", err)\n\t}\n\n\t// Filter the iterator by namespace\n\tf := func(raw any) bool {\n\t\tv, ok := raw.(*structs.CSIVolume)\n\t\tif !ok {\n\t\t\treturn false\n\t\t}\n\t\treturn v.Namespace != namespace && strings.HasPrefix(v.ID, prefix)\n\t}\n\n\twrap := memdb.NewFilterIterator(iter, f)\n\treturn wrap, nil\n}\n\n// CSIVolumesByIDPrefix supports search. Caller should snapshot if it wants to\n// also denormalize the plugins. If using a prefix with the wildcard namespace,\n// the results will not use the index prefix.","sourceCodeStart":2728,"sourceCodeEnd":2764,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L2728-L2764","documentation":"Returned by StateStore.CSIVolumesByPluginID when txn.Get(TableCSIVolumes, \"plugin_id\", pluginID) fails to build the iterator over volumes registered with a given CSI plugin. Note the namespace filter is applied afterwards via a SliceIterator filter, so this error is purely the memdb Get on the plugin_id index failing.","triggerScenarios":"Listing volumes by plugin ID (e.g. node/plugin status endpoints) while the \"plugin_id\" index Get errors — read-txn invalidity or schema mismatch.","commonSituations":"Custom builds with altered indexes; corrupted in-memory store after a bug or OOM; upgrade path issues.","solutions":["Inspect the wrapped %v message in the server log","Restart the agent to rebuild state from the raft log","Ensure all servers run an unmodified, matching Nomad version","Report with debug bundle if reproducible"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"iter, _, err := api.CSIVolumesByPluginID(prefix, pluginID)\nif err != nil && strings.Contains(err.Error(), \"volume lookup failed\") {\n    // plugin_id index Get failed: retry after server restart\n}","preventionTips":["Monitor state store health via server metrics","Restart agents promptly on memdb errors","Keep forks in sync with upstream schema changes","Verify plugin ID exists via nomad plugin status before listing its volumes"],"tags":["state-store","memdb","csi","plugin","nomad"],"backgroundTag":"memdb-lookup-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}