hashicorp/nomad · error

failed to decode task state from 'simple-all' entry: %v

Error message

failed to decode task state from 'simple-all' entry: %v

What it means

upgradeTaskBucket decodes the task's 0.8 state from the 'simple-all' key using msgpack. If the bytes do not decode into taskRunnerState08, the task's state cannot be migrated; the caller logs a warning and drops the task bucket.

Source

Thrown at client/state/upgrade.go:291

			continue
		}

		if !bytes.Equal(k, []byte("simple-all")) {
			// value is non-nil: delete unexpected entry
			logger.Warn("deleting unexpected task state entry",
				"key", string(k), "value_bytes", len(v),
			)

			if err := cur.Delete(); err != nil {
				return nil, fmt.Errorf("error delting unexpected task key %q: %v", string(k), err)
			}
			continue
		}

		// Decode simple-all
		simpleFound = true
		if err := codec.NewDecoderBytes(v, structs.MsgpackHandle).Decode(&trState); err != nil {
			return nil, fmt.Errorf("failed to decode task state from 'simple-all' entry: %v", err)
		}
	}

	if !simpleFound {
		return nil, fmt.Errorf("task state entry not found")
	}

	return &trState, nil
}

// upgradeOldAllocMutable upgrades Nomad 0.8 alloc runner state.
func upgradeOldAllocMutable(tx *boltdd.Tx, allocID string, oldBytes []byte) error {
	var oldMutable allocRunnerMutableState08
	err := codec.NewDecoderBytes(oldBytes, structs.MsgpackHandle).Decode(&oldMutable)
	if err != nil {
		return err
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Back up the client data_dir, stop nomad, and remove the affected alloc/task state so migration can skip it
2. Let the client re-run the task from scratch; dropped task state means the task restarts under 0.9+
3. Verify disk health and that the boltdb file was not restored mid-write
4. Reproduce with a msgpack dump of the entry and report upstream if the payload looks valid

Example fix

// before: unreadable legacy state blocks migration of that task
// after: recover by clearing stale client state
// nomad stop; mv <data_dir>/client/state.db <data_dir>/client/state.db.bak; nomad start

// Go: accept the drop but alert
if err := UpgradeAllocs(logger, tx); err != nil && strings.Contains(err.Error(), "failed to decode task state") {
    logger.Warn("corrupt legacy task state will be dropped and task restarted", "err", err)
}

When it happens

Trigger: 'simple-all' entry contains bytes that fail msgpack decode into taskRunnerState08 (truncated write, different codec handle, binary corruption) during UpgradeAllocs from 0.8 state.

Common situations: Disk corruption or unclean shutdown while running Nomad 0.8; restoring a boltdb file from an inconsistent backup; state written by a dev build with a different msgpack schema.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• error deleting invalid task state for task %q: %v
• error deleting unexpected task bucket %q: %v
• error delting unexpected task key %q: %v
• task state entry not found
• failed to read dynamic plugin registry state: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/f793fb864f866f00. Report an issue: GitHub.