hashicorp/nomad · error

failed to decode task state from handle: %v

Error message

failed to decode task state from handle: %v

What it means

RecoverTask decodes the driver-specific state stored in the task handle into a MockTaskState via handle.GetDriverState. If the stored bytes cannot be unmarshaled (corrupt, empty, or written by an incompatible driver version), the driver logs and returns this wrapped error, refusing to restore the task.

Source

Thrown at drivers/mock/driver.go:368

	}

	return &drivers.Fingerprint{
		Attributes:        attrs,
		Health:            health,
		HealthDescription: desc,
	}
}

func (d *Driver) RecoverTask(handle *drivers.TaskHandle) error {
	if handle == nil {
		return fmt.Errorf("handle cannot be nil")
	}

	// Unmarshall the driver state and create a new handle
	var taskState MockTaskState
	if err := handle.GetDriverState(&taskState); err != nil {
		d.logger.Error("failed to decode task state from handle", "error", err, "task_id", handle.Config.ID)
		return fmt.Errorf("failed to decode task state from handle: %v", err)
	}

	taskState.Command.parseDurations()
	if taskState.ExecCommand != nil {
		taskState.ExecCommand.parseDurations()
	}

	// Correct the run_for time based on how long it has already been running
	now := time.Now()
	if !taskState.StartedAt.IsZero() {
		taskState.Command.runForDuration = taskState.Command.runForDuration - now.Sub(taskState.StartedAt)

		if taskState.ExecCommand != nil {
			taskState.ExecCommand.runForDuration = taskState.ExecCommand.runForDuration - now.Sub(taskState.StartedAt)
		}
	}

	// Recreate the taskHandle. Because there's no real running process, we'll

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Inspect handle state: it likely doesn't match MockTaskState; ensure the handle came from the mock driver's own PersistState.
2. If the state is corrupt after a crash or upgrade, let the task fail and restart it rather than recovering.
3. Keep driver and client versions consistent when rolling upgrades; clear stale data-dir state for the affected task.
4. In tests, build the handle by running the driver (StartTask) instead of hand-assembling driver state.

Example fix

// before
handle := drivers.NewTaskHandle("mock") // empty state
plugin.RecoverTask(handle)
// after
handle := drivers.NewTaskHandle("mock")
h := driver.StartTask(cfg)
// recover using the handle produced by the driver itself

var probe MockTaskState
if err := json.Unmarshal(rawDriverState, &probe); err != nil {
    return fmt.Errorf("handle state is not valid MockTaskState; resubmit task instead of recovering")
}

func isMockState(h *drivers.TaskHandle) bool {
    var s MockTaskState
    return h != nil && h.GetDriverState(&s) == nil
}

if err := d.RecoverTask(h); err != nil {
    if strings.Contains(err.Error(), "failed to decode task state") {
        // state corrupt/incompatible: abandon recovery, restart task fresh
        return d.StartTask(cfg)
    }
    return err
}

When it happens

Trigger: Calling RecoverTask with a handle whose driver state JSON does not match MockTaskState — e.g. state persisted by a different driver, truncated on disk, or schema changed between Nomad versions.

Common situations: Upgrading Nomad/mock driver across a schema change; hand-crafted handles in tests missing required fields; data-dir corruption after a crash; attaching the wrong driver's handle.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• handle cannot be nil
• failed to decode task state from handle: %v
• no exec command is configured
• unable to process ACLToken: %w
• failed to read device manager plugin state: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/4748cd1aab607175. Report an issue: GitHub.