hashicorp/nomad · error
updating job summary failed: %v
Error message
updating job summary failed: %v
What it means
Thrown when txn.Insert("job_summary", jobSummary) fails after the summary fields were updated. The insert writes the whole JobSummary object back to the memdb 'job_summary' table; failure aborts the transaction and the allocation-driven summary update is lost.
Source
Thrown at nomad/state/state_store.go:6164
s.logger.Error("invalid old client status for allocation",
"alloc_id", existingAlloc.ID, "client_status", existingAlloc.ClientStatus)
}
summaryChanged = true
}
jobSummary.Summary[alloc.TaskGroup] = tgSummary
if summaryChanged {
jobSummary.ModifyIndex = index
s.updatePluginWithJobSummary(index, jobSummary, alloc, txn)
// Update the indexes table for job summary
if err := txn.Insert("index", &IndexEntry{"job_summary", index}); err != nil {
return fmt.Errorf("index update failed: %v", err)
}
if err := txn.Insert("job_summary", jobSummary); err != nil {
return fmt.Errorf("updating job summary failed: %v", err)
}
}
return nil
}
// updatePluginForTerminalAlloc updates the CSI plugins for an alloc when the
// allocation is updated or inserted with a terminal server status.
func (s *StateStore) updatePluginForTerminalAlloc(index uint64, alloc *structs.Allocation,
txn *txn) error {
if !alloc.ServerTerminalStatus() {
return nil
}
tg := alloc.Job.LookupTaskGroup(alloc.TaskGroup)
for _, t := range tg.Tasks {
if t.CSIPluginConfig != nil {View on GitHub (pinned to 482b49bf1a)
Solutions
- Retry the raft operation; transient memdb failures usually resolve
- Monitor server memory; OOM pressure can cause memdb allocation failures
- Restore the state store from a consistent snapshot if corruption is suspected
- Capture server logs and file an issue with HashiCorp if reproducible
Defensive patterns
Strategy: retry
Try / catch
try {
await op();
} catch (e) {
if (String(e).includes('updating job summary failed') && isTransient(e)) return retryWithBackoff(op);
// persistent: capture logs, consider snapshot restore
throw e;
} Prevention
- Ensure servers have adequate memory headroom for the state store
- Restore state only from consistent snapshots taken with nomad operator snapshot save
- Watch server logs for early memdb warnings
- Avoid running servers at OOM threshold
When it happens
Trigger: The memdb insert of the JobSummary object fails during the allocation update path — essentially only due to internal memdb errors (memory exhaustion, transaction/table corruption), since the object was just fetched and mutated.
Common situations: Severely memory-constrained Nomad servers; corrupted state store after a failed restore; rare go-memdb runtime failures.
Related errors
- index update failed: %v
- csi_plugin lookup error: %s %v
- csi_plugins insert error: %v
- csi_plugins lookup failed: %v
- csi_plugins lookup error %s: %v
AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04).
Data as JSON: /api/errors/d805c6b61d96affc.
Report an issue: GitHub.