hashicorp/nomad · error

failed to lookup job deployments: %v

Error message

failed to lookup job deployments: %v

What it means

downgradedJobForPlacement (used for disconnected-client / lost-restore handling) looks up the job's deployments via StateStore.DeploymentsByJobID to find the latest promoted or canaried version. A state store error aborts the placement with this wrapped error.

Source

Thrown at scheduler/generic_sched.go:456

	destructive := make([]reconciler.PlacementResult, 0, len(result.DestructiveUpdate))
	for _, p := range result.DestructiveUpdate {
		s.queuedAllocs[p.TaskGroup().Name] += 1
		destructive = append(destructive, p)
	}
	return s.computePlacements(destructive, place, result.TaskGroupAllocNameIndexes)
}

// downgradedJobForPlacement returns the previous stable version of the job for
// downgrading a placement for non-canaries
func (s *GenericScheduler) downgradedJobForPlacement(p reconciler.PlacementResult) (string, *structs.Job, error) {
	ns, jobID := s.job.Namespace, s.job.ID
	tgName := p.TaskGroup().Name

	// find deployments and use the latest promoted or canaried version
	deployments, err := s.state.DeploymentsByJobID(nil, ns, jobID, false)
	if err != nil {
		return "", nil, fmt.Errorf("failed to lookup job deployments: %v", err)
	}

	sort.Slice(deployments, func(i, j int) bool {
		return deployments[i].JobVersion > deployments[j].JobVersion
	})

	for _, d := range deployments {
		// It's unexpected to have a recent deployment that doesn't contain the TaskGroup; as all allocations
		// should be destroyed. In such cases, attempt to find the deployment for that TaskGroup and hopefully
		// we will kill it soon.  This is a defensive measure, have not seen it in practice
		//
		// Zero dstate.DesiredCanaries indicates that the TaskGroup allocates were updated in-place without using canaries.
		if dstate := d.TaskGroups[tgName]; dstate != nil && (dstate.Promoted || dstate.DesiredCanaries == 0) {
			job, err := s.state.JobByIDAndVersion(nil, ns, jobID, d.JobVersion)
			return d.ID, job, err
		}
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check the wrapped underlying error in server logs
2. Retry — the eval will be rescheduled
3. Verify state store and raft health; restore from snapshot if needed
4. Upgrade Nomad if the error recurs under disconnected-client scenarios

nomad deployment list // confirm deployment records are queryable before relying on reconnect flows

// Inspect the wrapped error in server logs; rely on Nomad's eval retry
nomad deployment list
nomad eval list | grep -i failed

When it happens

Trigger: StateStore.DeploymentsByJobID(nil, ns, jobID, false) errors during computePlacements when a placement needs a downgraded job version (client reconnected after disconnection).

Common situations: State store instability while handling disconnected clients rejoining, heavy deployment churn, server resource pressure.

Related errors

• failed to get job deployment %q: %v
• deployment promotion cannot be undone
• eval broker is enabled; eval broker must be paused to delete
• deployment id not found: %q
• deployment %q references unknown job %q

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/5eebe86ea8c7a68a. Report an issue: GitHub.