{"record":{"id":"906040d62c3a81b1","repo":"hashicorp/nomad","slug":"restart-of-an-alloc-that-should-not-run","errorCode":null,"errorMessage":"restart of an alloc that should not run","messagePattern":"restart of an alloc that should not run","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/alloc_runner.go","lineNumber":1429,"sourceCode":"// RestartRunning restarts all tasks that are currently running.\nfunc (ar *allocRunner) RestartRunning(event *structs.TaskEvent) error {\n\treturn ar.restartTasks(context.TODO(), event, false, false)\n}\n\n// RestartAll restarts all tasks in the allocation, including dead ones. They\n// will restart following their lifecycle order.\nfunc (ar *allocRunner) RestartAll(event *structs.TaskEvent) error {\n\t// Restart the taskCoordinator to allow dead tasks to run again.\n\tar.taskCoordinator.Restart()\n\treturn ar.restartTasks(context.TODO(), event, false, true)\n}\n\n// restartTasks restarts all task runners concurrently.\nfunc (ar *allocRunner) restartTasks(ctx context.Context, event *structs.TaskEvent, failure bool, force bool) error {\n\n\t// ensure we are not trying to restart an alloc that is terminal\n\tif !ar.shouldRun() {\n\t\treturn fmt.Errorf(\"restart of an alloc that should not run\")\n\t}\n\n\twaitCh := make(chan struct{})\n\tvar err *multierror.Error\n\tvar errMutex sync.Mutex\n\n\t// run alloc task restart hooks\n\tar.taskRestartHooks()\n\n\tgo func() {\n\t\tvar wg sync.WaitGroup\n\t\tdefer close(waitCh)\n\t\tfor tn, tr := range ar.tasks {\n\t\t\twg.Add(1)\n\t\t\tgo func(taskName string, taskRunner *taskrunner.TaskRunner) {\n\t\t\t\tdefer wg.Done()\n\n\t\t\t\tvar e error","sourceCodeStart":1411,"sourceCodeEnd":1447,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/alloc_runner.go#L1411-L1447","documentation":"restartTasks refuses to restart any tasks when the allocation is in a terminal state (shouldRun() is false — alloc dead/failed/cancelled). This guard prevents reviving tasks of an alloc that the scheduler has already concluded.","triggerScenarios":"RestartTask/RestartAll/RestartRunning invoked (API `nomad alloc restart`, task event handlers) on an alloc whose client status means it should no longer run: alloc stopped, job stopped/failed, or rescheduled away.","commonSituations":"Operators restarting allocs that were just stopped by a `nomad job stop` or GC; race where the alloc became terminal between listing and restart; scripts retrying restarts on a dead alloc.","solutions":["Check the alloc status with `nomad alloc status <id>`; only restart allocs in a running/pending state","If the work is needed, redeploy the job (`nomad job run`) so a new live allocation is created instead of restarting a terminal one","In automation, re-fetch alloc state immediately before issuing restart and treat this error as 'alloc terminal — skip'","Avoid restarting allocs belonging to stopped/old job versions; target the latest deployment's allocations"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Only restart allocs that are not terminal\nalloc, _, _ := client.Allocations().Info(allocID, nil)\nif alloc.ClientStatus == \"complete\" || alloc.ClientStatus == \"failed\" ||\n   alloc.DesiredStatus == \"stop\" || alloc.DesiredStatus == \"evict\" {\n  return fmt.Errorf(\"alloc %s is terminal; redeploy instead\", allocID)\n}","typeGuard":null,"tryCatchPattern":"err := client.Allocations().Restart(allocID, \"\", nil)\nif err != nil && strings.Contains(err.Error(), \"should not run\") {\n  // alloc terminal: run `nomad job run` to create a new allocation instead\n}","preventionTips":["Re-fetch alloc state immediately before issuing restarts","Do not restart allocs after `nomad job stop` or GC — redeploy instead","In retry loops, treat this error as terminal and stop retrying","Track alloc desired/client status in automation before lifecycle actions"],"tags":["lifecycle","alloc-runner","terminal-state","api"],"backgroundTag":"restart-terminal-alloc","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}