What causes "restart of an alloc that should not run" in hashicorp/nomad?

Check the alloc status with `nomad alloc status `; only restart allocs in a running/pending state

restart of an alloc that should not run

Error message

restart of an alloc that should not run

What it means

restartTasks refuses to restart any tasks when the allocation is in a terminal state (shouldRun() is false — alloc dead/failed/cancelled). This guard prevents reviving tasks of an alloc that the scheduler has already concluded.

Source

Thrown at client/allocrunner/alloc_runner.go:1429

// RestartRunning restarts all tasks that are currently running.
func (ar *allocRunner) RestartRunning(event *structs.TaskEvent) error {
	return ar.restartTasks(context.TODO(), event, false, false)
}

// RestartAll restarts all tasks in the allocation, including dead ones. They
// will restart following their lifecycle order.
func (ar *allocRunner) RestartAll(event *structs.TaskEvent) error {
	// Restart the taskCoordinator to allow dead tasks to run again.
	ar.taskCoordinator.Restart()
	return ar.restartTasks(context.TODO(), event, false, true)
}

// restartTasks restarts all task runners concurrently.
func (ar *allocRunner) restartTasks(ctx context.Context, event *structs.TaskEvent, failure bool, force bool) error {

	// ensure we are not trying to restart an alloc that is terminal
	if !ar.shouldRun() {
		return fmt.Errorf("restart of an alloc that should not run")
	}

	waitCh := make(chan struct{})
	var err *multierror.Error
	var errMutex sync.Mutex

	// run alloc task restart hooks
	ar.taskRestartHooks()

	go func() {
		var wg sync.WaitGroup
		defer close(waitCh)
		for tn, tr := range ar.tasks {
			wg.Add(1)
			go func(taskName string, taskRunner *taskrunner.TaskRunner) {
				defer wg.Done()

				var e error

View on GitHub (pinned to 482b49bf1a)

Solutions

Check the alloc status with `nomad alloc status <id>`; only restart allocs in a running/pending state
If the work is needed, redeploy the job (`nomad job run`) so a new live allocation is created instead of restarting a terminal one
In automation, re-fetch alloc state immediately before issuing restart and treat this error as 'alloc terminal — skip'
Avoid restarting allocs belonging to stopped/old job versions; target the latest deployment's allocations

Defensive patterns

Strategy: validation

Validate before calling

// Only restart allocs that are not terminal
alloc, _, _ := client.Allocations().Info(allocID, nil)
if alloc.ClientStatus == "complete" || alloc.ClientStatus == "failed" ||
   alloc.DesiredStatus == "stop" || alloc.DesiredStatus == "evict" {
  return fmt.Errorf("alloc %s is terminal; redeploy instead", allocID)
}

Try / catch

err := client.Allocations().Restart(allocID, "", nil)
if err != nil && strings.Contains(err.Error(), "should not run") {
  // alloc terminal: run `nomad job run` to create a new allocation instead
}

Prevention

Re-fetch alloc state immediately before issuing restarts
Do not restart allocs after `nomad job stop` or GC — redeploy instead
In retry loops, treat this error as terminal and stop retrying
Track alloc desired/client status in automation before lifecycle actions

When it happens

Trigger: RestartTask/RestartAll/RestartRunning invoked (API `nomad alloc restart`, task event handlers) on an alloc whose client status means it should no longer run: alloc stopped, job stopped/failed, or rescheduled away.

Common situations: Operators restarting allocs that were just stopped by a `nomad job stop` or GC; race where the alloc became terminal between listing and restart; scripts retrying restarts on a dead alloc.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/906040d62c3a81b1. Report an issue: GitHub.