{"record":{"id":"9a5b79a697092b74","repo":"hashicorp/nomad","slug":"job-summary-insert-failed-v","errorCode":null,"errorMessage":"job summary insert failed: %v","messagePattern":"job summary insert failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":2008,"sourceCode":"\t\t\t\t\tpSummary.Children.Pending--\n\t\t\t\t\tpSummary.Children.Dead++\n\t\t\t\t\tmodified = true\n\t\t\t\tcase structs.JobStatusRunning:\n\t\t\t\t\tpSummary.Children.Running--\n\t\t\t\t\tpSummary.Children.Dead++\n\t\t\t\t\tmodified = true\n\t\t\t\tcase structs.JobStatusDead:\n\t\t\t\tdefault:\n\t\t\t\t\treturn fmt.Errorf(\"unknown old job status %q\", job.Status)\n\t\t\t\t}\n\n\t\t\t\tif modified {\n\t\t\t\t\t// Update the modify index\n\t\t\t\t\tpSummary.ModifyIndex = index\n\n\t\t\t\t\t// Insert the summary\n\t\t\t\t\tif err := txn.Insert(\"job_summary\", pSummary); err != nil {\n\t\t\t\t\t\treturn fmt.Errorf(\"job summary insert failed: %v\", err)\n\t\t\t\t\t}\n\t\t\t\t\tif err := txn.Insert(\"index\", &IndexEntry{\"job_summary\", index}); err != nil {\n\t\t\t\t\t\treturn fmt.Errorf(\"index update failed: %v\", err)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// Delete the job\n\tif err := txn.Delete(\"jobs\", existing); err != nil {\n\t\treturn fmt.Errorf(\"job delete failed: %v\", err)\n\t}\n\tif err := txn.Insert(\"index\", &IndexEntry{\"jobs\", index}); err != nil {\n\t\treturn fmt.Errorf(\"index update failed: %v\", err)\n\t}\n\n\t// Delete the job versions","sourceCodeStart":1990,"sourceCodeEnd":2026,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L1990-L2026","documentation":"DeleteJobTxn (nomad/state/state_store.go:2008) wraps a failure from txn.Insert(\"job_summary\", pSummary) after the parent job summary counters were modified while deleting a child job. The summary write failed so the deregistration transaction aborts and rolls back; the wrapped %v carries the underlying txn.Insert cause.","triggerScenarios":"Deleting a dispatched/periodic child whose parent summary was modified, and txn.Insert on the job_summary table fails (invalid summary object, MemDB error, aborted transaction).","commonSituations":"In-memory DB issues on the server applying the deregistration; corrupted JobSummary entries restored from a bad snapshot; concurrent modifications breaking assumptions within the txn.","solutions":["Retry the deregistration; the failed txn rolled back","Read the wrapped %v error in server logs to find the root cause","Restart the server agent to rebuild state from Raft","If tied to a snapshot restore, re-verify snapshot integrity"],"exampleFix":"// before\nerr := s.DeleteJobTxn(idx, ns, jobID, txn)\n// after\nif err := s.DeleteJobTxn(idx, ns, jobID, txn); err != nil {\n    if strings.Contains(err.Error(), \"job summary insert failed\") {\n        return retry(func() error { return s.DeleteJobTxn(idx, ns, jobID, txn) })\n    }\n    return err\n}","handlingStrategy":"retry","validationCode":"// ensure the parent summary exists and is well-formed before child delete\nif job.ParentID != \"\" {\n    sum, err := stateStore.JobSummaryByID(nil, ns, job.ParentID)\n    if err != nil { return err }\n    if sum != nil && sum.Children == nil {\n        return fmt.Errorf(\"parent summary for %s has nil Children; state is inconsistent\", job.ParentID)\n    }\n}","typeGuard":"func validSummary(x interface{}) bool { s, ok := x.(*structs.JobSummary); return ok && s != nil && s.Children != nil }","tryCatchPattern":"if err := s.DeleteJobTxn(idx, ns, jobID, txn); err != nil {\n    if strings.Contains(err.Error(), \"job summary insert failed\") {\n        return retry(3, backoff, func() error { return s.DeleteJobTxn(idx, ns, jobID, txn) })\n    }\n    return err\n}","preventionTips":["Retry failed deregistrations — txns roll back atomically","Check server logs for the wrapped root cause","Restore only verified snapshots","Monitor server memory pressure during dispatch/stop storms"],"tags":["state-store","job-summary","raft"],"backgroundTag":"memdb-transaction-insert-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}