hashicorp/nomad · error

unknown old job status %q

Error message

unknown old job status %q

What it means

DeleteJobTxn (nomad/state/state_store.go:1999) returns this when decrementing the parent job's summary counters for a dead child whose recorded job.Status is neither JobStatusRunning nor JobStatusDead. The switch over the old status hits its default branch, meaning the stored status string is unexpected — usually a corrupt or hand-edited state entry.

Source

Thrown at nomad/state/state_store.go:1999

		// job was removed
		if summaryRaw != nil {
			existing := summaryRaw.(*structs.JobSummary)
			pSummary := existing.Copy()
			if pSummary.Children != nil {

				modified := false
				switch job.Status {
				case structs.JobStatusPending:
					pSummary.Children.Pending--
					pSummary.Children.Dead++
					modified = true
				case structs.JobStatusRunning:
					pSummary.Children.Running--
					pSummary.Children.Dead++
					modified = true
				case structs.JobStatusDead:
				default:
					return fmt.Errorf("unknown old job status %q", job.Status)
				}

				if modified {
					// Update the modify index
					pSummary.ModifyIndex = index

					// Insert the summary
					if err := txn.Insert("job_summary", pSummary); err != nil {
						return fmt.Errorf("job summary insert failed: %v", err)
					}
					if err := txn.Insert("index", &IndexEntry{"job_summary", index}); err != nil {
						return fmt.Errorf("index update failed: %v", err)
					}
				}
			}
		}
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check the job's stored Status in the snapshot/raft data; it must be "running" or "dead" at delete time
2. Confirm all server agents run a compatible Nomad version (no mixed-version state restores)
3. If state is corrupt, restore from a known-good Raft snapshot
4. Report the exact %q status value if it comes from stock Nomad — it indicates a state-store bug

Example fix

// before (state-store switch)
case structs.JobStatusDead:
default:
    return fmt.Errorf("unknown old job status %q", job.Status)
// after (defensive caller check before deleting a child)
if job.Status != structs.JobStatusRunning && job.Status != structs.JobStatusDead {
    return fmt.Errorf("refusing to delete child job %s with unexpected status %q", job.ID, job.Status)
}
return s.DeleteJobTxn(idx, ns, jobID, txn)

// verify the child's stored status is one the summary logic understands
switch job.Status {
case structs.JobStatusRunning, structs.JobStatusDead:
    // safe to delete
default:
    return fmt.Errorf("child job %s has unsupported status %q; fix state before delete", job.ID, job.Status)
}

func knownJobStatus(s string) bool {
    return s == structs.JobStatusRunning || s == structs.JobStatusDead || s == structs.JobStatusPending
}

if err := s.DeleteJobTxn(idx, ns, jobID, txn); err != nil {
    if strings.HasPrefix(err.Error(), "unknown old job status") {
        // corrupt/foreign status value: quarantine the job and investigate state
        return investigateStateCorruption(err)
    }
    return err
}

When it happens

Trigger: Deleting a child job (ParentID != "") whose parent summary exists and whose stored Status value is not exactly "running" or "dead" (e.g. "pending" persisted where it should not be, or an unknown status from a version mismatch).

Common situations: State restored from an older/newer Nomad version with different status vocabulary; corrupted Raft logs; custom tooling writing job status directly into state snapshots.

Related errors

• job summary for job %q in namespace %q is not present
• detected corrupted token within the state store: missing rol
• unexpected bucket missing %q
• error getting job summary: %v
• unable to create job summary: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/8a84f3ee40ef8911. Report an issue: GitHub.