{"record":{"id":"9861e3560c00a50c","repo":"hashicorp/nomad","slug":"task-group-volume-claim-insert-failed-v","errorCode":null,"errorMessage":"Task group volume claim insert failed: %v","messagePattern":"Task group volume claim insert failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"nomad/state/state_store_task_group_volume_claims.go","lineNumber":58,"sourceCode":"\t\texisting = existingRaw.(*structs.TaskGroupHostVolumeClaim)\n\t}\n\n\tif existing != nil {\n\t\t// do allocation ID and volume ID match?\n\t\tif existing.ClaimedByAlloc(claim) {\n\t\t\treturn nil\n\t\t}\n\n\t\tclaim.CreateIndex = existing.CreateIndex\n\t\tclaim.ModifyIndex = index\n\t} else {\n\t\tclaim.CreateIndex = index\n\t\tclaim.ModifyIndex = index\n\t}\n\n\t// Insert the claim into the table.\n\tif err := txn.Insert(TableTaskGroupHostVolumeClaim, claim); err != nil {\n\t\treturn fmt.Errorf(\"Task group volume claim insert failed: %v\", err)\n\t}\n\n\t// Perform the index table update to mark the new insert.\n\tif err := txn.Insert(tableIndex, &IndexEntry{TableTaskGroupHostVolumeClaim, index}); err != nil {\n\t\treturn fmt.Errorf(\"index update failed: %v\", err)\n\t}\n\n\treturn nil\n}\n\n// GetTaskGroupHostVolumeClaim returns a volume claim that matches the namespace,\n// job id and task group name (there can be only one)\nfunc (s *StateStore) GetTaskGroupHostVolumeClaim(ws memdb.WatchSet, namespace, jobID, taskGroupName, volumeID string) (*structs.TaskGroupHostVolumeClaim, error) {\n\ttxn := s.db.ReadTxn()\n\n\twatchCh, existing, err := txn.FirstWatch(TableTaskGroupHostVolumeClaim, indexID, namespace, jobID, taskGroupName, volumeID)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"Task group volume claim lookup failed: %v\", err)","sourceCodeStart":40,"sourceCodeEnd":76,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store_task_group_volume_claims.go#L40-L76","documentation":"This error wraps a memdb transaction failure that occurred while inserting a TaskGroupHostVolumeClaim row into the state store. txn.Insert fails only for low-level store problems (invalid object, txn corruption, schema mismatch), since claim-vs-existing conflicts are handled as updates earlier in the function. It surfaces during UpsertTaskGroupHostVolumeClaim or when sticky volume claims are reconciled from an allocation update.","triggerScenarios":"txn.Insert(TableTaskGroupHostVolumeClaim, claim) returns an error during UpsertTaskGroupHostVolumeClaim or updateStickyVolumeClaimsFromAlloc; practically only when the in-memory store is corrupt or the object violates the table schema.","commonSituations":"Corrupted Raft/state store requiring restart; a Nomad bug or version-mismatch writing an incompatible claim struct; disk/memory pressure corrupting memdb.","solutions":["Retry the Raft write; transient txn issues usually clear on retry","Check server logs for preceding memdb/schema errors to identify the underlying %v cause","Restart the Nomad server if the state store is believed corrupt (state is reloaded from Raft snapshots)","Upgrade Nomad if running a version with known volume-claim bugs"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := store.UpsertTaskGroupHostVolumeClaim(idx, claims); err != nil {\n    if strings.Contains(err.Error(), \"insert failed\") {\n        // low-level store error: backoff then retry once\n        return retryAfterBackoff()\n    }\n    return err\n}","preventionTips":["Keep the Nomad server healthy (disk, memory) so memdb writes don't fail","Watch server logs for memdb errors as an early-warning signal","Avoid running external tools against the server's data directory","Pin supported Nomad versions"],"tags":["nomad","state-store","memdb","volume-claims"],"backgroundTag":"state-store-write-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}