hashicorp/nomad · error

failed to get tainted nodes for job '%s': %v

Error message

failed to get tainted nodes for job '%s': %v

What it means

SystemScheduler.computeJobAllocs failed to determine which tainted (drained/down) nodes hold the job's allocations, after successfully listing them. The wrapped error carries the state-store lookup cause.

Source

Thrown at scheduler/scheduler_system.go:240

	s.stack.SetJob(job)
	s.stack.SetSchedulerConfiguration(schedConfig.WithNodePool(pool))
	return nil
}

// computeJobAllocs is used to reconcile differences between the job,
// existing allocations and node status to update the allocations.
func (s *SystemScheduler) computeJobAllocs() error {
	// Lookup the allocations by JobID
	ws := memdb.NewWatchSet()
	allocs, err := s.state.AllocsByJob(ws, s.eval.Namespace, s.eval.JobID, true)
	if err != nil {
		return fmt.Errorf("failed to get allocs for job '%s': %v", s.eval.JobID, err)
	}

	// Determine the tainted nodes containing job allocs
	tainted, err := taintedNodes(s.state, allocs)
	if err != nil {
		return fmt.Errorf("failed to get tainted nodes for job '%s': %v", s.eval.JobID, err)
	}

	// Update the allocations which are in pending/running state on tainted
	// nodes to lost.
	updateNonTerminalAllocsToLost(s.plan, tainted, allocs)

	// Split out terminal allocations
	live, term := structs.SplitTerminalAllocs(allocs)

	// Diff the required and existing allocations
	nr := reconciler.NewNodeReconciler(s.deployment)
	reconciliationResult := nr.Compute(s.job, s.nodes, s.notReadyNodes, tainted,
		live, term)
	if s.logger.IsDebug() {
		s.logger.Debug("reconciled current state with desired state", reconciliationResult.Fields()...)
	}

	// Update the stored deployment

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check server logs for the node query failure
2. Rely on the scheduler's retry loop once the read recovers

When it happens

Trigger: Thrown at scheduler/scheduler_system.go:240 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/eab307eab01d6e4a. Report an issue: GitHub.