hashicorp/nomad · error

failed to start command path=%q --- args=%q: %v

Error message

failed to start command path=%q --- args=%q: %v

What it means

Launch finally starts the child process (e.childCmd.Start) — optionally under network isolation — and wraps any Start failure with the command path and args. This is the definitive 'task process could not be launched' error in the universal executor.

Source

Thrown at drivers/shared/executor/executor.go:429

	absPath, err := lookupBin(command.TaskDir, command.Cmd)
	if err != nil {
		return nil, err
	}

	if err := makeExecutable(absPath); err != nil {
		return nil, err
	}

	path := absPath

	// Set the commands arguments
	e.childCmd.Path = path
	e.childCmd.Args = append([]string{e.childCmd.Path}, command.Args...)
	e.childCmd.Env = e.command.Env

	// Start the process
	if err = withNetworkIsolation(e.childCmd.Start, command.NetworkIsolation); err != nil {
		return nil, fmt.Errorf("failed to start command path=%q --- args=%q: %v", path, e.childCmd.Args, err)
	}

	// Run the runningFunc hook after the process starts
	if err := running(); err != nil {
		return nil, err
	}

	// Wait on the task process
	go e.wait()
	return &ProcessState{Pid: e.childCmd.Process.Pid, ExitCode: -1, Time: time.Now()}, nil
}

// Exec a command inside a container for exec and java drivers.
func (e *UniversalExecutor) Exec(deadline time.Time, name string, args []string) ([]byte, int, error) {
	ctx, cancel := context.WithDeadline(context.Background(), deadline)
	defer cancel()

	if cleanup, err := e.setSubCmdCgroup(&e.childCmd, e.command.StatsCgroup()); err != nil {

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Read path and args in the message and verify the binary exists/executable at exactly that path
2. If NetworkIsolation is set, check CNI/bridge setup and CAP_NET_ADMIN on the client node
3. Check chroot/task dir contents for the expected binary (missing files in chroot list)
4. Test the command manually as the nomad user; inspect the wrapped %v cause for errno
5. Raise process/fd limits if EAGAIN/fork failures appear

Example fix

// before
// exec: "nomad-task": no such file or directory (binary not in chroot)
// after
// copy binary into the task dir or fix driver config: command = "/local/nomad-task" with artifact staging

func preflightLaunch(path string, args []string) error {
  fi, err := os.Stat(path)
  if err != nil { return fmt.Errorf("ENOENT: %w", err) }
  if fi.Mode()&0o111 == 0 { return fmt.Errorf("EACCES: %s not executable", path) }
  for _, a := range args { if a == "" { return errors.New("empty arg") } }
  return nil
}

if err := launchTask(); err != nil {
  if strings.Contains(err.Error(), "failed to start command path=") {
    var eerr *exec.Error
    if errors.As(err, &eerr) {
      return fmt.Errorf("fix driver command config for %s: %w", eerr.Name, err)
    }
    if strings.Contains(err.Error(), "network") {
      return checkCNIAndRetry()
    }
  }
  return err
}

When it happens

Trigger: withNetworkIsolation(e.childCmd.Start, ...) fails: fork/exec error (ENOENT, EACCES, ENOEXEC), netns setup failure (creating/joining the network namespace fails), or process resource limits.

Common situations: Wrong driver 'command' path, binary missing from image/chroot, architecture mismatch (ENOEXEC), network isolation misconfiguration (invalid bridge, missing CAP_NET_ADMIN, CNI plugin failure), fork limits under heavy load.

Related errors

• failed to start command: %v
• Exec: %w
• failed to identify user %v: %v
• command is not present
• not a terminal

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/2f0e43ef03f4bd2a. Report an issue: GitHub.