{"record":{"id":"d805c6b61d96affc","repo":"hashicorp/nomad","slug":"updating-job-summary-failed-v","errorCode":null,"errorMessage":"updating job summary failed: %v","messagePattern":"updating job summary failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":6164,"sourceCode":"\t\t\ts.logger.Error(\"invalid old client status for allocation\",\n\t\t\t\t\"alloc_id\", existingAlloc.ID, \"client_status\", existingAlloc.ClientStatus)\n\t\t}\n\t\tsummaryChanged = true\n\t}\n\tjobSummary.Summary[alloc.TaskGroup] = tgSummary\n\n\tif summaryChanged {\n\t\tjobSummary.ModifyIndex = index\n\n\t\ts.updatePluginWithJobSummary(index, jobSummary, alloc, txn)\n\n\t\t// Update the indexes table for job summary\n\t\tif err := txn.Insert(\"index\", &IndexEntry{\"job_summary\", index}); err != nil {\n\t\t\treturn fmt.Errorf(\"index update failed: %v\", err)\n\t\t}\n\n\t\tif err := txn.Insert(\"job_summary\", jobSummary); err != nil {\n\t\t\treturn fmt.Errorf(\"updating job summary failed: %v\", err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// updatePluginForTerminalAlloc updates the CSI plugins for an alloc when the\n// allocation is updated or inserted with a terminal server status.\nfunc (s *StateStore) updatePluginForTerminalAlloc(index uint64, alloc *structs.Allocation,\n\ttxn *txn) error {\n\n\tif !alloc.ServerTerminalStatus() {\n\t\treturn nil\n\t}\n\n\ttg := alloc.Job.LookupTaskGroup(alloc.TaskGroup)\n\tfor _, t := range tg.Tasks {\n\t\tif t.CSIPluginConfig != nil {","sourceCodeStart":6146,"sourceCodeEnd":6182,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L6146-L6182","documentation":"Thrown when txn.Insert(\"job_summary\", jobSummary) fails after the summary fields were updated. The insert writes the whole JobSummary object back to the memdb 'job_summary' table; failure aborts the transaction and the allocation-driven summary update is lost.","triggerScenarios":"The memdb insert of the JobSummary object fails during the allocation update path — essentially only due to internal memdb errors (memory exhaustion, transaction/table corruption), since the object was just fetched and mutated.","commonSituations":"Severely memory-constrained Nomad servers; corrupted state store after a failed restore; rare go-memdb runtime failures.","solutions":["Retry the raft operation; transient memdb failures usually resolve","Monitor server memory; OOM pressure can cause memdb allocation failures","Restore the state store from a consistent snapshot if corruption is suspected","Capture server logs and file an issue with HashiCorp if reproducible"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  await op();\n} catch (e) {\n  if (String(e).includes('updating job summary failed') && isTransient(e)) return retryWithBackoff(op);\n  // persistent: capture logs, consider snapshot restore\n  throw e;\n}","preventionTips":["Ensure servers have adequate memory headroom for the state store","Restore state only from consistent snapshots taken with nomad operator snapshot save","Watch server logs for early memdb warnings","Avoid running servers at OOM threshold"],"tags":["nomad","state-store","memdb","job-summary"],"backgroundTag":"state-store-insert-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}