hashicorp/nomad · error

executor Shutdown failed: %v

Error message

executor Shutdown failed: %v

What it means

StopTask calls handle.exec.Shutdown(signal, timeout) to gracefully stop the task's executor, and this error wraps any failure from that shutdown. As a special case, if the plugin client has already exited (the executor process is gone), StopTask returns nil instead — so this error only surfaces when the executor is still connected but failed to shut down within the signal/timeout contract.

Source

Thrown at drivers/java/driver.go:640

	case <-ctx.Done():
		return
	case <-d.ctx.Done():
		return
	case ch <- result:
	}
}

func (d *Driver) StopTask(taskID string, timeout time.Duration, signal string) error {
	handle, ok := d.tasks.Get(taskID)
	if !ok {
		return drivers.ErrTaskNotFound
	}

	if err := handle.exec.Shutdown(signal, timeout); err != nil {
		if handle.pluginClient.Exited() {
			return nil
		}
		return fmt.Errorf("executor Shutdown failed: %v", err)
	}

	return nil
}

func (d *Driver) DestroyTask(taskID string, force bool) error {
	handle, ok := d.tasks.Get(taskID)
	if !ok {
		return drivers.ErrTaskNotFound
	}

	if handle.IsRunning() && !force {
		return fmt.Errorf("cannot destroy running task")
	}

	if !handle.pluginClient.Exited() {
		if err := handle.exec.Shutdown("", 0); err != nil {
			handle.logger.Error("destroying executor failed", "error", err)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Read the wrapped %v error to see whether it was a timeout or a shutdown RPC failure
2. Retry StopTask; if it still fails, DestroyTask with force=true will forcibly shut the executor down
3. Increase the task's kill_timeout so slow JVM shutdown hooks can complete
4. Inspect the executor logs for a hung or deadlocked JVM and fix the application's shutdown hooks
5. Verify the executor plugin connection is healthy (nomad node status, client logs)

Example fix

// before: 5s default may be too short for JVM shutdown hooks
kill_timeout = "5s"
// after: give the JVM time to run shutdown hooks
job "app" {
  group "g" {
    task "java-app" {
      kill_timeout = "30s"
    }
  }
}

// ensure generous kill_timeout in the task spec so Shutdown has room
// kill_timeout = "30s"

err := driver.StopTask(taskID, "SIGINT", 30*time.Second)
if err != nil {
    if strings.Contains(err.Error(), "executor Shutdown failed") && !driver.IsRunning(taskID) {
        return nil // executor actually stopped despite the error
    }
    // fall back to forced destroy
    return driver.DestroyTask(taskID, true)
}

When it happens

Trigger: Calling Driver.StopTask where exec.Shutdown returns an error AND handle.pluginClient.Exited() is false — i.e., the executor is alive but refused or failed to shut down (signal not handled, timeout expired while the JVM ignored SIGTERM, executor internal error).

Common situations: Java applications that trap SIGTERM and hang past the kill_timeout, a wedged JVM (thread dump deadlock, GC thrash), or executor plugin communication issues during client shutdown.

Related errors

• QEMU graceful shutdown is unsupported on the Windows platfor
• default_pid_mode must be %q or %q, got %q
• default_ipc_mode must be %q or %q, got %q
• allow_caps configured with capabilities not supported by sys
• pid_mode must be %q or %q, got %q

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/d7d81003ac03705b. Report an issue: GitHub.