hashicorp/nomad · error

executor Shutdown failed: %v

Error message

executor Shutdown failed: %v

What it means

raw_exec's StopTask asks the executor to gracefully shut down the task (signal + timeout). If exec.Shutdown returns an error and the executor plugin has NOT exited, the driver surfaces 'executor Shutdown failed'. Notably, if the plugin already exited the error is swallowed and nil is returned.

Source

Thrown at drivers/rawexec/driver.go:537

	case <-ctx.Done():
		return
	case <-d.ctx.Done():
		return
	case ch <- result:
	}
}

func (d *Driver) StopTask(taskID string, timeout time.Duration, signal string) error {
	handle, ok := d.tasks.Get(taskID)
	if !ok {
		return drivers.ErrTaskNotFound
	}

	if err := handle.exec.Shutdown(signal, timeout); err != nil {
		if handle.pluginClient.Exited() {
			return nil
		}
		return fmt.Errorf("executor Shutdown failed: %v", err)
	}

	// Wait for handle to finish
	<-handle.doneCh

	// Kill executor
	handle.pluginClient.Kill()

	return nil
}

func (d *Driver) DestroyTask(taskID string, force bool) error {
	handle, ok := d.tasks.Get(taskID)
	if !ok {
		return drivers.ErrTaskNotFound
	}

	if handle.IsRunning() && !force {

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Wait for the kill_timeout to elapse — Nomad will force-kill (SIGKILL) after graceful shutdown fails
2. Check whether the task handles the shutdown signal; add a signal handler or adjust the task's kill_signal/kill_timeout
3. Check client logs for executor plugin errors; restart the Nomad client agent if the executor is wedged
4. Upgrade Nomad if the executor repeatedly fails to signal the process

if err := driver.StopTask(id, timeout, sig); err != nil {
    // plugin may have died; check handle.pluginClient.Exited() semantics
    log.Warn("graceful shutdown failed, waiting for force kill", "err", err)
    <-doneCh // Nomad will SIGKILL after kill_timeout
}

When it happens

Trigger: Calling StopTask while the executor plugin is alive but Shutdown fails: the underlying process ignores the shutdown signal, the executor IPC call errors/times out, or the executor process is wedged.

Common situations: Tasks that ignore SIGTERM/SIGINT and hang past the kill_timeout; frozen or deadlocked task processes; executor plugin communication failures on a loaded client.

Related errors

• failed to reattach to executor: %v
• failed to create executor: %v
• executor: error waiting on process: %v
• only one of cgroups_v1_override and cgroups_v2_override may
• work_dir must be an absolute path

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/851af102523a7c10. Report an issue: GitHub.