hashicorp/nomad · error

task with ID %q already started

Error message

task with ID %q already started

What it means

This error is thrown by the Docker driver's StartTask when a task with the same ID is already registered in the driver's in-memory task store. The driver refuses to start a duplicate task to prevent two Docker containers from being created for one allocation/task ID. It indicates a caller-side lifecycle bug: StartTask was invoked for a task that was never restored or cleaned up.

Source

Thrown at drivers/docker/driver.go:332

	go h.run()

	return nil
}

func loggingIsEnabled(driverCfg *DriverConfig, taskCfg *drivers.TaskConfig) bool {
	if driverCfg.DisableLogCollection {
		return false
	}
	if taskCfg.StderrPath == os.DevNull && taskCfg.StdoutPath == os.DevNull {
		return false
	}
	return true
}

func (d *Driver) StartTask(cfg *drivers.TaskConfig) (*drivers.TaskHandle, *drivers.DriverNetwork, error) {
	if _, ok := d.tasks.Get(cfg.ID); ok {
		return nil, nil, fmt.Errorf("task with ID %q already started", cfg.ID)
	}

	var driverConfig TaskConfig

	if err := cfg.DecodeDriverConfig(&driverConfig); err != nil {
		return nil, nil, fmt.Errorf("failed to decode driver config: %v", err)
	}

	if driverConfig.Image == "" {
		return nil, nil, fmt.Errorf("image name required for docker driver")
	}

	driverConfig.Image = strings.TrimPrefix(driverConfig.Image, "https://")

	driverConfig.ImagePullTimeout = getValue(driverConfig.ImagePullTimeout, d.config.ImagePullTimeout)

	handle := drivers.NewTaskHandle(taskHandleVersion)
	handle.Config = cfg

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check d.tasks (the in-memory task store) for the task ID and remove the stale entry via the driver's task cleanup/DestroyTask path before restarting
2. Verify the caller is not issuing duplicate StartTask calls for the same allocation; dedupe on the client/scheduler side
3. Restart the Nomad client or reload the driver plugin to clear the in-memory task map if the task is genuinely gone (e.g. container already removed)
4. If the task should be reattached instead of started, use the driver's RecoverTask path rather than StartTask

Example fix

// before
d.tasks.Set(task.ID, handle)
...
// duplicate restart attempt
handle, net, err := driver.StartTask(cfg) // -> already started
// after
if d.tasks.Get(task.ID) != nil { // container handle already exists
    if _, err := driver.RecoverTask(&drivers.TaskRecoveryConfig{TaskID: task.ID, Handle: oldHandle}); err != nil {
        // handle recovery error
    }
} else {
    handle, net, err = driver.StartTask(cfg)
}

// caller-side: ensure task ID not already dispatched to this driver
if driverTaskHandles[cfg.ID] != nil {
    return fmt.Errorf("task %q already started; recover instead of start", cfg.ID)
}

func isTaskAlreadyStartedErr(err error) bool {
    return err != nil && strings.Contains(err.Error(), "already started")
}

if _, _, err := driver.StartTask(cfg); err != nil {
    if isTaskAlreadyStartedErr(err) {
        // reattach via RecoverTask instead of retrying StartTask
        return driver.RecoverTask(&drivers.TaskRecoveryConfig{TaskID: cfg.ID})
    }
    return err
}

When it happens

Trigger: Calling StartTask with a drivers.TaskConfig whose cfg.ID matches a task previously started (and still tracked) by the same driver instance — e.g. double-dispatch of a task start event, a retry after a partial failure where the task handle was already registered, or re-running StartTask during driver restore/reconnect without the task having been removed.

Common situations: Nomad client restarts where task state was restored but the plugin still tracks the task; client retrying a StartTask RPC after a network timeout even though the first call succeeded; a leaked task from a previous failed StartTask that never reached cleanup; plugins kept alive across allocation updates with stale task entries.

Related errors

• failed to reattach to docker logger process: %v
• failed to launch docker logger plugin: %v
• failed to launch docker logger process %s: %v
• failed to get docker client: %w
• failed to decode driver config: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/743a18b1a41b8b05. Report an issue: GitHub.