{"record":{"id":"d557c121dbb626d4","repo":"hashicorp/nomad","slug":"volume-row-conversion-error","errorCode":null,"errorMessage":"volume row conversion error","messagePattern":"volume row conversion error","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":2884,"sourceCode":"\treturn iter, nil\n}\n\n// CSIVolumeClaim updates the volume's claim count and allocation list\nfunc (s *StateStore) CSIVolumeClaim(index uint64, now int64, namespace, id string, claim *structs.CSIVolumeClaim) error {\n\ttxn := s.db.WriteTxnMsgT(structs.CSIVolumeClaimRequestType, index)\n\tdefer txn.Abort()\n\n\trow, err := txn.First(TableCSIVolumes, \"id\", namespace, id)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"volume lookup failed: %s: %v\", id, err)\n\t}\n\tif row == nil {\n\t\treturn fmt.Errorf(\"volume not found: %s\", id)\n\t}\n\n\torig, ok := row.(*structs.CSIVolume)\n\tif !ok {\n\t\treturn fmt.Errorf(\"volume row conversion error\")\n\t}\n\n\tvar alloc *structs.Allocation\n\tif claim.State == structs.CSIVolumeClaimStateTaken {\n\t\talloc, err = s.allocByIDImpl(txn, nil, claim.AllocationID)\n\t\tif err != nil {\n\t\t\ts.logger.Error(\"AllocByID failed\", \"error\", err)\n\t\t\treturn fmt.Errorf(structs.ErrUnknownAllocationPrefix)\n\t\t}\n\t\tif alloc == nil {\n\t\t\ts.logger.Error(\"AllocByID failed to find alloc\", \"alloc_id\", claim.AllocationID)\n\t\t}\n\t}\n\n\tvolume, err := s.csiVolumeDenormalizePluginsTxn(txn, orig.Copy())\n\tif err != nil {\n\t\treturn err\n\t}","sourceCodeStart":2866,"sourceCodeEnd":2902,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L2866-L2902","documentation":"Returned by Nomad's state store when a CSI volume claim update (CSIVolumeClaim) finds a row in the CSIVolumes table that cannot be type-asserted to *structs.CSIVolume. The memdb row store returned an object of an unexpected type, so the store refuses to mutate it rather than panic. This indicates internal state corruption or a schema/version mismatch, not a user API misuse.","triggerScenarios":"Calling state store CSIVolumeClaim (via volume claim RPC) when the row stored under TableCSIVolumes for the requested namespace/id is not a *structs.CSIVolume — typically after a corrupted or hand-edited BoltDB/memdb snapshot, a downgrade across Nomad versions that changed the stored row type, or a code bug inserting a different struct under the volume table.","commonSituations":"Restoring a raft snapshot from a different Nomad version; mixing state_store data across major upgrades; custom patched Nomad builds writing foreign row types; corrupted state store after crash.","solutions":["Check the Nomad server version consistency across the cluster and ensure the snapshot/raft data matches that version","Inspect the state store entry for the volume via `nomad volume status <id>` and server logs to confirm the row is corrupt","Delete and re-register the affected CSI volume (nomad volume deregister, then nomad volume register)","File an issue with HashiCorp with the server log and snapshot details — this is an internal invariant failure"],"exampleFix":"// Not caller-fixable; state-store-side hardening\n// before\norig, ok := row.(*structs.CSIVolume)\nif !ok {\n\treturn fmt.Errorf(\"volume row conversion error\")\n}\n// after\norig, ok := row.(*structs.CSIVolume)\nif !ok {\n\treturn fmt.Errorf(\"volume row conversion error: got %T\", row)\n}","handlingStrategy":"try-catch","validationCode":"// Go caller via API: pre-check the volume exists and reads cleanly\n_, err := client.CSIVolumes().Get(volumeID)\nif err != nil {\n\treturn fmt.Errorf(\"volume %s unreadable, skipping claim: %w\", volumeID, err)\n}","typeGuard":"func asCSIVolume(row interface{}) (*structs.CSIVolume, bool) {\n\tv, ok := row.(*structs.CSIVolume)\n\treturn v, ok\n}","tryCatchPattern":"err := client.CSIVolumes().Claim(claim)\nvar apiErr *api.QueryError\nif errors.As(err, &apiErr) && strings.Contains(err.Error(), \"conversion error\") {\n\t// internal corruption: escalate to operator, do not retry blindly\n}","preventionTips":["Keep all Nomad servers on the same version","Never hand-edit or restore snapshots across incompatible versions","Monitor server state store health and take regular raft snapshots","Treat this error as a bug report trigger, not a retryable condition"],"tags":["go","state-store","csi","nomad"],"backgroundTag":"csi-volume-row-corruption","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}