hashicorp/nomad · error

failed to get job %q: %v

Error message

failed to get job %q: %v

What it means

GenericScheduler.process looks up the evaluation's job with StateStore.JobByID. If the state store lookup returns an error (not merely a missing job), processing aborts with this wrapped error. This is a state-store failure, distinct from the job simply not existing.

Source

Thrown at scheduler/generic_sched.go:210

	if planFailure {
		s.blocked.TriggeredBy = structs.EvalTriggerMaxPlans
		s.blocked.StatusDescription = sstructs.DescBlockedEvalMaxPlan
	} else {
		s.blocked.StatusDescription = sstructs.DescBlockedEvalFailedPlacements
	}

	return s.planner.CreateEval(s.blocked)
}

// process is wrapped in retryMax to iteratively run the handler until we have no
// further work or we've made the maximum number of attempts.
func (s *GenericScheduler) process() (bool, error) {
	// Lookup the Job by ID
	var err error
	ws := memdb.NewWatchSet()
	s.job, err = s.state.JobByID(ws, s.eval.Namespace, s.eval.JobID)
	if err != nil {
		return false, fmt.Errorf("failed to get job %q: %v", s.eval.JobID, err)
	}

	numTaskGroups := 0
	stopped := s.job.Stopped()
	if !stopped {
		numTaskGroups = len(s.job.TaskGroups)
	}
	s.queuedAllocs = make(map[string]int, numTaskGroups)
	s.followUpEvals = nil

	// Create a plan
	s.plan = s.eval.MakePlan(s.job)

	if !s.batch {
		// Get any existing deployment
		s.deployment, err = s.state.LatestDeploymentByJobID(ws, s.eval.Namespace, s.eval.JobID)
		if err != nil {
			return false, fmt.Errorf("failed to get job deployment %q: %v", s.eval.JobID, err)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Inspect the wrapped underlying error for the root cause
2. Retry — Nomad reschedules eval processing; transient failures usually self-heal
3. Check server health (memory/disk/raft) and restore from snapshot if corruption is suspected
4. Verify the eval's namespace/JobID are sane; garbage eval payloads can point to deeper bugs

nomad job status <job-id> // confirm the job is queryable and the cluster is healthy

// Look up the wrapped error in server logs and verify cluster health
nomad eval list | grep -i failed
nomad operator raft list-peers

When it happens

Trigger: StateStore.JobByID(ws, eval.Namespace, eval.JobID) returns an error during process() — state store/memdb failure, closed or aborted snapshot, internal index error.

Common situations: Nomad server state store instability, raft contention, resource exhaustion, or version-specific state store bugs during eval handling.

Related errors

• eval broker is enabled; eval broker must be paused to delete
• failed to get job's allocations: %v
• failed to lookup node ID %q: %v
• failed to get job deployment %q: %v
• failed to get allocs for job '%s': %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/ec50dbf11ae7d6ce. Report an issue: GitHub.