hashicorp/nomad · error

task with ID %q already started

Error message

task with ID %q already started

What it means

StartTask rejects a task whose ID is already tracked by this driver instance, meaning a taskHandle already exists in d.tasks for cfg.ID. This is an internal invariant guard: the same task ID must not be started twice. It usually surfaces after a client-side retry that didn't notice the earlier start succeeded.

Source

Thrown at drivers/rawexec/driver.go:397

				break
			}
		}

		if !found {
			envList = append(envList, k+"="+v)
		}
	}
	sort.Strings(envList)
	return envList
}

func (d *Driver) StartTask(cfg *drivers.TaskConfig) (*drivers.TaskHandle, *drivers.DriverNetwork, error) {
	if !d.config.Enabled {
		return nil, nil, errDisabledDriver
	}

	if _, ok := d.tasks.Get(cfg.ID); ok {
		return nil, nil, fmt.Errorf("task with ID %q already started", cfg.ID)
	}

	var driverConfig TaskConfig
	if err := cfg.DecodeDriverConfig(&driverConfig); err != nil {
		return nil, nil, fmt.Errorf("failed to decode driver config: %v", err)
	}

	driverConfig.OverrideCgroupV2 = cgroupslib.CustomPathCG2(driverConfig.OverrideCgroupV2)

	if err := driverConfig.validate(); err != nil {
		return nil, nil, fmt.Errorf("failed driver config validation: %v", err)
	}

	if err := d.Validate(*cfg); err != nil {
		return nil, nil, fmt.Errorf("failed driver config validation: %v", err)
	}

	d.logger.Info("starting task", "driver_cfg", hclog.Fmt("%+v", driverConfig))

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Let the client reconcile: the existing handle is still valid, so the client should call it via RecoverTask/its existing handle instead of starting again
2. Restart the nomad client agent if its task map is stale/duplicated
3. Report to Nomad if reproducible — this indicates a client-side race and should not happen under normal operation
4. Check for multiple nomad client processes accidentally running against the same data_dir

// caller: check driver doesn't already track the task before StartTask
// (client side) ensure only one StartTask per task ID is in flight
if started[taskID] { return existingHandle, nil }

func isAlreadyStarted(err error) bool {
  var pe *strings.Reader // message-based guard
  return err != nil && strings.Contains(err.Error(), "already started")
}

h, net, err := d.StartTask(cfg)
if err != nil && strings.Contains(err.Error(), "already started") {
  // treat as success-ish: retrieve existing handle via RecoverTask
  return d.RecoverTask(cfg.ID)
}

When it happens

Trigger: Calling StartTask with a TaskConfig whose ID is already present in the driver's task map — e.g. Nomad client retrying StartTask after an RPC timeout, or RecoverTask/StartTask racing on the same task.

Common situations: RPC retries from client to driver plugin after network hiccups; duplicate task IDs from a corrupted client state; running a single nomad client process with duplicated alloc assignments.

Related errors

• failed to set driver state: %v
• only one of cgroups_v1_override and cgroups_v2_override may
• work_dir must be an absolute path
• task with ID %q already started
• task with ID %q already started

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/a38fd89e0a9e1cec. Report an issue: GitHub.