What causes "task %q not started yet." in hashicorp/nomad?

Wait until `nomad alloc status ` shows the task as running before exec'ing

task %q not started yet.

Error message

task %q not started yet.

What it means

execImpl checks that the task not only exists but has actually started (taskState.StartedAt is non-zero) before wiring up the exec stream. If the task exists in the allocation but has not yet been started by the client runner, the endpoint returns HTTP 404 with this message. The task is known but not yet runnable.

Source

Thrown at client/alloc_endpoint.go:359

	allocState, err := a.c.GetAllocState(req.AllocID)
	if err != nil {
		code := new(int64(500))
		if nstructs.IsErrUnknownAllocation(err) {
			code = new(int64(404))
		}

		return code, err
	}

	// Check that the task is there
	taskState := allocState.TaskStates[req.Task]
	if taskState == nil {
		return new(int64(400)), fmt.Errorf("unknown task name %q", req.Task)
	}

	if taskState.StartedAt.IsZero() {
		return new(int64(404)), fmt.Errorf("task %q not started yet.", req.Task)
	}

	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	h := ar.GetTaskExecHandler(req.Task)
	if h == nil {
		return new(int64(404)), fmt.Errorf("task %q is not running.", req.Task)
	}

	err = h(ctx, req.Cmd, req.Tty, newExecStream(decoder, encoder))
	if err != nil {
		code := new(int64(500))
		return code, err
	}

	return nil, nil
}

View on GitHub (pinned to 482b49bf1a)

Solutions

Wait until `nomad alloc status <alloc-id>` shows the task as running before exec'ing
Poll the allocation/task state via the API until StartedAt is set
Retry the exec command with backoff after placement
Investigate why the task has not started (driver errors, image pull failures, template blocking)

Example fix

// before
client.Exec(allocID, "web", []string{"sh"}) // immediately after job run
// after
for {
  alloc, _, _ := client.Allocations().Info(allocID, nil)
  ts := alloc.TaskStates["web"]
  if ts != nil && !ts.StartedAt.IsZero() { break }
  time.Sleep(time.Second)
}
client.Exec(allocID, "web", []string{"sh"})

Defensive patterns

Strategy: retry

Validate before calling

alloc, _, _ := client.Allocations().Info(allocID, nil)
ts := alloc.TaskStates[taskName]
ready := ts != nil && !ts.StartedAt.IsZero()

Type guard

func taskStarted(alloc *api.Allocation, task string) bool {
  ts := alloc.TaskStates[task]
  return ts != nil && !ts.StartedAt.IsZero()
}

Try / catch

var code int64
for i := 0; i < 10; i++ {
  code, err = client.Allocations().Exec(ctx, alloc, taskName, tty, false, cmd, stdin, stdout, stderr, nil, nil)
  if err == nil || code != 404 || !strings.Contains(fmt.Sprint(err), "not started yet") { break }
  time.Sleep(2 * time.Second)
}

Prevention

Wait for task state Running before exec (poll alloc info)
Use `nomad job dispatch`-style readiness gates in CI
Add backoff-retry around exec after fresh deployment

When it happens

Trigger: Invoking exec immediately after the allocation is placed but before the task process launches (image pull, driver startup, etc.), or exec'ing a task stuck in pending state after a restart/reschedule.

Common situations: CI scripts that run `nomad alloc exec` immediately after `nomad job run` without waiting for the allocation to become running; slow container image downloads; tasks waiting on vault tokens or consul health before start.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/7e34ea73d2a7f7c9. Report an issue: GitHub.