hashicorp/nomad · error

Failed to restart task %q: %w

Error message

Failed to restart task %q: %w

What it means

Per-task restart via client.Allocations().Restart(alloc, taskName, nil) failed; the error is wrapped as 'Failed to restart task <name>: <cause>' and returned through an errgroup so all tasks restart concurrently.

Source

Thrown at command/job_restart.go:921

	var restarts multierror.Group
	for task := range c.tasks.Items() {
		if !alloc.HasTask(task) {
			continue
		}

		c.Ui.Output(fmt.Sprintf(
			"    %s: Restarting task %q in allocation %q for group %q",
			formatTime(time.Now()),
			task,
			shortAllocID,
			alloc.TaskGroup,
		))

		restarts.Go(func(taskName string) func() error {
			return func() error {
				err := c.client.Allocations().Restart(&api.Allocation{ID: alloc.ID}, taskName, nil)
				if err != nil {
					return fmt.Errorf("Failed to restart task %q: %w", taskName, err)
				}
				return nil
			}
		}(task))
	}
	return restarts.Wait().ErrorOrNil()
}

// stopAlloc stops an allocation and blocks until the replacement allocation is
// running.
func (c *JobRestartCommand) stopAlloc(alloc AllocationListStubWithJob) error {
	shortAllocID := limit(alloc.ID, c.length)

	c.Ui.Output(fmt.Sprintf(
		"    %s: Rescheduling allocation %q for group %q",
		formatTime(time.Now()),
		shortAllocID,
		alloc.TaskGroup,

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Inspect nomad alloc status <alloc-id> to confirm the task is running
  2. Check the client node logs for the driver-level restart error
  3. Verify the alloc ID is current (job may have been rescheduled)
  4. Retry once the node/agent is reachable
Defensive patterns

Strategy: try-catch

Validate before calling

alloc, _, err := client.Allocations().Info(alloc.ID, nil)
if err != nil {
    return err
}
taskState, ok := alloc.TaskStates[taskName]
if !ok || taskState.State != "running" {
    return fmt.Errorf("task %s not running in alloc %s", taskName, alloc.ID)
}

Type guard

func taskRunning(a *api.Allocation, task string) bool {
    ts, ok := a.TaskStates[task]
    return ok && ts.State == "running"
}

Try / catch

err := c.client.Allocations().Restart(&api.Allocation{ID: alloc.ID}, task, nil)
if err != nil {
    if isRetryable(err) { return retryWithBackoff() }
    return fmt.Errorf("non-retryable restart failure: %w", err)
}

Prevention

When it happens

Trigger: The Restart API call for a specific task returns an error — e.g. alloc not running, task not found in the alloc, driver error, or RPC failure to the client node.

Common situations: Task already dead/failed so restart is invalid; node disconnected during restart; stale alloc ID after a reschedule; insufficient ACL permissions.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/d148e3b4226ab43c. Report an issue: GitHub.