What causes "exec not possible, client status of allocation %s is %s" in hashicorp/nomad?

Verify the allocation is still running: `nomad alloc status ` and check ClientStatus is 'run' before exec'ing.

exec not possible, client status of allocation %s is %s

Error message

exec not possible, client status of allocation %s is %s

What it means

Nomad's AllocExec handler rejects an exec request when the target allocation's client status is terminal (completed, failed, or lost), meaning the task's process tree no longer exists on the node. The server checks alloc.ClientTerminalStatus() before opening the exec stream and returns HTTP 400 wrapped in this message. It is thrown because exec requires a live task to attach to.

Source

Thrown at nomad/client_alloc_endpoint.go:546

		return
	}
	if err != nil {
		handleStreamResultError(err, nil, encoder)
		return
	}

	// Check node read permissions
	if aclObj, err := a.srv.ResolveACL(&args); err != nil {
		handleStreamResultError(err, nil, encoder)
		return
	} else if !aclObj.AllowNsOp(alloc.Namespace, acl.NamespaceCapabilityAllocExec) {
		// client ultimately checks if AllocNodeExec is required
		handleStreamResultError(structs.ErrPermissionDenied, nil, encoder)
		return
	}

	if alloc.ClientTerminalStatus() {
		handleStreamResultError(fmt.Errorf("exec not possible, client status of allocation %s is %s", alloc.ID, alloc.ClientStatus),
			new(int64(http.StatusBadRequest)), encoder)
		return
	}

	// Handle job ID if requested.
	if args.JobID != "" {
		// Verify job exists.
		job, err := snap.JobByID(nil, args.Namespace, args.JobID)
		if err != nil {
			handleStreamResultError(err,
				new(int64(http.StatusInternalServerError)), encoder)
			return
		}
		if job == nil {
			handleStreamResultError(
				fmt.Errorf("job %s not found in namespace %s", args.JobID, args.Namespace),
				new(int64(http.StatusNotFound)), encoder)
			return

View on GitHub (pinned to 482b49bf1a)

Solutions

Verify the allocation is still running: `nomad alloc status <alloc-id>` and check ClientStatus is 'run' before exec'ing.
Re-run the job or use `nomad job run` to create a new live allocation, then exec into that.
For batch work, capture logs instead: `nomad alloc logs <alloc-id>` works on terminal allocs.
In automation, retry exec with backoff or handle terminal status by skipping/relaunching.

Example fix

// before
exec(allocID) // alloc may be terminal
// after
const st = getAlloc(allocID)
if (st.clientStatus !== 'run') { logs(allocID); return } // skip exec for terminal alloc
exec(allocID)

Defensive patterns

Strategy: validation

Validate before calling

const status = await nomad.allocStatus(allocID)
if (['complete','failed','lost'].includes(status.clientStatus)) {
  throw new SkipExecError(`alloc ${allocID} terminal (${status.clientStatus}); use logs instead`)
}
return exec(allocID)

Type guard

const isRunnable = (a) => a != null && a.clientStatus === 'run'

Prevention

Check alloc ClientStatus with `nomad alloc status` immediately before exec.
Treat batch/parameterized jobs as exec-unfriendly; prefer logs.
In automation, poll until ClientStatus is 'run' with a timeout before exec.
Alert on exec attempts against terminal allocs to catch race conditions.

When it happens

Trigger: Calling `nomad alloc exec <allocID> ...` (or the AllocExec RPC) with an allocation whose ClientStatus is 'complete', 'failed', or 'lost'.

Common situations: Scripts/race conditions where the job finished between listing allocations and exec'ing into it; exec'ing into a batch/parameterized job's finished alloc; targeting an alloc from a failed deployment; node restart marking allocs lost.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/598ad96f00bf801c. Report an issue: GitHub.