hashicorp/nomad · error

executor: error waiting on process: %v

Error message

executor: error waiting on process: %v

What it means

handleWait wraps any error from the executor's Wait call into the task's ExitResult.Err. It indicates the executor plugin failed while waiting on the QEMU process, not that QEMU itself exited with a bad code.

Source

Thrown at drivers/qemu/driver.go:835

// GetAbsolutePath returns the absolute path of the passed binary by resolving
// it in the path and following symlinks.
func GetAbsolutePath(bin string) (string, error) {
	lp, err := exec.LookPath(bin)
	if err != nil {
		return "", fmt.Errorf("failed to resolve path to %q executable: %v", bin, err)
	}

	return filepath.EvalSymlinks(lp)
}

func (d *Driver) handleWait(ctx context.Context, handle *taskHandle, ch chan *drivers.ExitResult) {
	defer close(ch)
	var result *drivers.ExitResult
	ps, err := handle.exec.Wait(ctx)
	if err != nil {
		result = &drivers.ExitResult{
			Err: fmt.Errorf("executor: error waiting on process: %v", err),
		}
		// if process state is nil, we've probably been killed, so return a reasonable
		// exit state to the handlers
		if ps == nil {
			result.ExitCode = -1
			result.OOMKilled = false
		}
	} else {
		result = &drivers.ExitResult{
			ExitCode:  ps.ExitCode,
			Signal:    ps.Signal,
			OOMKilled: ps.OOMKilled,
		}
	}

	select {
	case <-ctx.Done():
	case <-d.ctx.Done():

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Read the wrapped %v error in the ExitResult / alloc task events for the root cause
  2. If ExitCode is -1 with process state nil, the task was likely killed externally — check host logs (OOM killer, SIGKILL)
  3. Restart the task/alloc; if the executor plugin is repeatedly failing, restart the Nomad client
Defensive patterns

Strategy: try-catch

Type guard

// Go: inspect ExitResult to distinguish killed vs failed wait
if res.ExitCode == -1 && res.Err != nil && strings.Contains(res.Err.Error(), "error waiting on process") {
    // executor wait failed, not a normal exit
}

Try / catch

res, err := ch.recv(ctx) // ExitResult channel from handleWait
if err != nil || (res != nil && res.Err != nil) {
    log.Printf("task wait failed: %v", res.Err)
}

Prevention

When it happens

Trigger: handle.exec.Wait(ctx) returns an error — e.g. the executor plugin crashed, the context was cancelled mid-wait, or IPC with the executor broke.

Common situations: Nomad client shutting down and cancelling wait contexts; executor plugin process killed; plugin client connection dropped mid-task.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/b51aa67e9d2572a8. Report an issue: GitHub.