{"record":{"id":"8a84f3ee40ef8911","repo":"hashicorp/nomad","slug":"unknown-old-job-status-q","errorCode":null,"errorMessage":"unknown old job status %q","messagePattern":"unknown old job status %q","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":1999,"sourceCode":"\t\t// job was removed\n\t\tif summaryRaw != nil {\n\t\t\texisting := summaryRaw.(*structs.JobSummary)\n\t\t\tpSummary := existing.Copy()\n\t\t\tif pSummary.Children != nil {\n\n\t\t\t\tmodified := false\n\t\t\t\tswitch job.Status {\n\t\t\t\tcase structs.JobStatusPending:\n\t\t\t\t\tpSummary.Children.Pending--\n\t\t\t\t\tpSummary.Children.Dead++\n\t\t\t\t\tmodified = true\n\t\t\t\tcase structs.JobStatusRunning:\n\t\t\t\t\tpSummary.Children.Running--\n\t\t\t\t\tpSummary.Children.Dead++\n\t\t\t\t\tmodified = true\n\t\t\t\tcase structs.JobStatusDead:\n\t\t\t\tdefault:\n\t\t\t\t\treturn fmt.Errorf(\"unknown old job status %q\", job.Status)\n\t\t\t\t}\n\n\t\t\t\tif modified {\n\t\t\t\t\t// Update the modify index\n\t\t\t\t\tpSummary.ModifyIndex = index\n\n\t\t\t\t\t// Insert the summary\n\t\t\t\t\tif err := txn.Insert(\"job_summary\", pSummary); err != nil {\n\t\t\t\t\t\treturn fmt.Errorf(\"job summary insert failed: %v\", err)\n\t\t\t\t\t}\n\t\t\t\t\tif err := txn.Insert(\"index\", &IndexEntry{\"job_summary\", index}); err != nil {\n\t\t\t\t\t\treturn fmt.Errorf(\"index update failed: %v\", err)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n","sourceCodeStart":1981,"sourceCodeEnd":2017,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L1981-L2017","documentation":"DeleteJobTxn (nomad/state/state_store.go:1999) returns this when decrementing the parent job's summary counters for a dead child whose recorded job.Status is neither JobStatusRunning nor JobStatusDead. The switch over the old status hits its default branch, meaning the stored status string is unexpected — usually a corrupt or hand-edited state entry.","triggerScenarios":"Deleting a child job (ParentID != \"\") whose parent summary exists and whose stored Status value is not exactly \"running\" or \"dead\" (e.g. \"pending\" persisted where it should not be, or an unknown status from a version mismatch).","commonSituations":"State restored from an older/newer Nomad version with different status vocabulary; corrupted Raft logs; custom tooling writing job status directly into state snapshots.","solutions":["Check the job's stored Status in the snapshot/raft data; it must be \"running\" or \"dead\" at delete time","Confirm all server agents run a compatible Nomad version (no mixed-version state restores)","If state is corrupt, restore from a known-good Raft snapshot","Report the exact %q status value if it comes from stock Nomad — it indicates a state-store bug"],"exampleFix":"// before (state-store switch)\ncase structs.JobStatusDead:\ndefault:\n    return fmt.Errorf(\"unknown old job status %q\", job.Status)\n// after (defensive caller check before deleting a child)\nif job.Status != structs.JobStatusRunning && job.Status != structs.JobStatusDead {\n    return fmt.Errorf(\"refusing to delete child job %s with unexpected status %q\", job.ID, job.Status)\n}\nreturn s.DeleteJobTxn(idx, ns, jobID, txn)","handlingStrategy":"validation","validationCode":"// verify the child's stored status is one the summary logic understands\nswitch job.Status {\ncase structs.JobStatusRunning, structs.JobStatusDead:\n    // safe to delete\ndefault:\n    return fmt.Errorf(\"child job %s has unsupported status %q; fix state before delete\", job.ID, job.Status)\n}","typeGuard":"func knownJobStatus(s string) bool {\n    return s == structs.JobStatusRunning || s == structs.JobStatusDead || s == structs.JobStatusPending\n}","tryCatchPattern":"if err := s.DeleteJobTxn(idx, ns, jobID, txn); err != nil {\n    if strings.HasPrefix(err.Error(), \"unknown old job status\") {\n        // corrupt/foreign status value: quarantine the job and investigate state\n        return investigateStateCorruption(err)\n    }\n    return err\n}","preventionTips":["Never write job status directly into state snapshots","Keep all servers on the same Nomad version","Validate Raft snapshots before restoring","Log the exact %q value to identify the corrupt source"],"tags":["state-store","job-summary","status","corruption"],"backgroundTag":"unexpected-enum-status-value","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}