hashicorp/nomad · error

error getting job summary: %v

Error message

error getting job summary: %v

What it means

Wraps a failure reading the job summary during deleteJobFromPlugins, invoked from DeleteJobTxn to strip the job's allocations from CSI plugins. JobSummaryByID errored, so the job delete cannot proceed and the transaction aborts.

Source

Thrown at nomad/state/state_store.go:1591

			return fmt.Errorf("csi_plugins delete error: %v", err)
		}
	} else {
		plug.ModifyIndex = index
		err := txn.Insert(TableCSIPlugins, plug)
		if err != nil {
			return fmt.Errorf("csi_plugins update error %s: %v", plug.ID, err)
		}
	}
	return nil
}

// deleteJobFromPlugins removes the allocations of this job from any plugins the job is
// running, possibly deleting the plugin if it's no longer in use. It's called in DeleteJobTxn
func (s *StateStore) deleteJobFromPlugins(index uint64, txn Txn, job *structs.Job) error {
	ws := memdb.NewWatchSet()
	summary, err := s.JobSummaryByID(ws, job.Namespace, job.ID)
	if err != nil {
		return fmt.Errorf("error getting job summary: %v", err)
	}

	allocs, err := s.AllocsByJob(ws, job.Namespace, job.ID, false)
	if err != nil {
		return fmt.Errorf("error getting allocations: %v", err)
	}

	type pair struct {
		pluginID string
		alloc    *structs.Allocation
	}

	plugAllocs := []*pair{}
	found := map[string]struct{}{}

	// Find plugins for allocs that belong to this job
	for _, a := range allocs {
		tg := a.Job.LookupTaskGroup(a.TaskGroup)

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Retry the job deletion.
  2. Check nomad job status and nomad alloc status for the job to see whether state is partially applied; stop remaining allocs first.
  3. Inspect the wrapped %v error for the underlying memdb cause.
  4. Upgrade Nomad — job-summary consistency bugs have been fixed in patch releases; as a last resort restore Raft state from backup.

Example fix

// before: force-purge fails with this error
$ nomad job stop -purge csi-job
// after: stop and wait for allocs to become terminal, then purge
$ nomad job stop csi-job
$ nomad job status csi-job   # wait: all allocs terminal
$ nomad job stop -purge csi-job
Defensive patterns

Strategy: try-catch

Validate before calling

// before purging a job with CSI volumes, confirm state is coherent
job, _, err := client.Jobs().Info(jobID, nil)
if err != nil { return err }
allocs, _, err := client.Jobs().Allocations(jobID, false, nil)
if err != nil { return err }
for _, a := range allocs {
    if a.ClientStatus != "complete" && a.ClientStatus != "failed" {
        return fmt.Errorf("alloc %s not terminal; stop job first", a.ID)
    }
}

Try / catch

_, _, err := client.Jobs().Deregister(jobID, true, nil)
if err != nil && strings.Contains(err.Error(), "error getting job summary") {
    // job state partially applied; retry after allocs settle, then force purge
    time.Sleep(10 * time.Second)
    _, _, err = client.Jobs().Deregister(jobID, true, nil)
}
return err

Prevention

When it happens

Trigger: nomad job stop / job deregistration for a job with CSI allocations; the internal ws lookup of the job's summary row fails (job summary row missing or memdb error).

Common situations: Seen when deleting jobs that had been partially mutated (e.g. summary rows affected by earlier failed ops or known Nomad job-summary bugs), or during state-store inconsistency.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/01f219240b558dc9. Report an issue: GitHub.