hashicorp/nomad · error

job summary lookup failed: %v

Error message

job summary lookup failed: %v

What it means

Returned by StateStore.nestedUpsertEval when txn.First on the 'job_summary' table fails while fetching the summary for the eval's (Namespace, JobID). The eval upsert needs the job summary to track queued allocations per task group; if this lookup errors the whole transaction is aborted.

Source

Thrown at nomad/state/state_store.go:3495

	// Lookup the evaluation
	existing, err := txn.First("evals", "id", eval.ID)
	if err != nil {
		return fmt.Errorf("eval lookup failed: %v", err)
	}

	// Update the indexes
	if existing != nil {
		eval.CreateIndex = existing.(*structs.Evaluation).CreateIndex
		eval.ModifyIndex = index
	} else {
		eval.CreateIndex = index
		eval.ModifyIndex = index
	}

	// Update the job summary
	summaryRaw, err := txn.First("job_summary", "id", eval.Namespace, eval.JobID)
	if err != nil {
		return fmt.Errorf("job summary lookup failed: %v", err)
	}
	if summaryRaw != nil {
		js := summaryRaw.(*structs.JobSummary).Copy()
		hasSummaryChanged := false
		for tg, num := range eval.QueuedAllocations {
			if summary, ok := js.Summary[tg]; ok {
				if summary.Queued != num {
					summary.Queued = num
					js.Summary[tg] = summary
					hasSummaryChanged = true
				}
			} else {
				s.logger.Error("unable to update queued for job and task group", "job_id", eval.JobID, "task_group", tg, "namespace", eval.Namespace)
			}
		}

		// Insert the job summary
		if hasSummaryChanged {

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Check the wrapped inner memdb error in the message
  2. Confirm the eval's namespace/job ID correspond to a registered job
  3. Restart servers to rebuild in-memory state if corruption is suspected
  4. Keep nomad versions consistent across the server cluster
Defensive patterns

Strategy: try-catch

Validate before calling

// Confirm the job (and its summary) exists before upserting evals
sum, err := state.JobSummaryByID(ws, eval.Namespace, eval.JobID)
if err != nil { return err }
// nil summary is tolerated by nestedUpsertEval; error means memdb failure

Type guard

if js, ok := raw.(*structs.JobSummary); ok { /* use js */ }

Try / catch

if err := state.UpsertEvals(idx, evals); err != nil {
	if strings.Contains(err.Error(), "job summary lookup failed") {
		// inspect inner memdb cause; retry once, else restart
	}
	return err
}

Prevention

When it happens

Trigger: Calling UpsertEvals when the job_summary read for eval.Namespace/eval.JobID errors in memdb — internal read failure rather than an absent summary (absent is handled by skipping the update).

Common situations: Evals for jobs whose summaries were concurrently removed; memdb read failure from schema mismatch after upgrade; namespaces/job IDs manipulated by custom tooling.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/55b72cbf9743e77d. Report an issue: GitHub.