hashicorp/nomad · warning

task state entry not found

Error message

task state entry not found

What it means

A 0.8 task bucket must contain a 'simple-all' key holding the task runner state. If iteration completes without finding it, upgradeTaskBucket returns this error; the caller warns and deletes the whole task bucket as invalid state.

Source

Thrown at client/state/upgrade.go:296

			logger.Warn("deleting unexpected task state entry",
				"key", string(k), "value_bytes", len(v),
			)

			if err := cur.Delete(); err != nil {
				return nil, fmt.Errorf("error delting unexpected task key %q: %v", string(k), err)
			}
			continue
		}

		// Decode simple-all
		simpleFound = true
		if err := codec.NewDecoderBytes(v, structs.MsgpackHandle).Decode(&trState); err != nil {
			return nil, fmt.Errorf("failed to decode task state from 'simple-all' entry: %v", err)
		}
	}

	if !simpleFound {
		return nil, fmt.Errorf("task state entry not found")
	}

	return &trState, nil
}

// upgradeOldAllocMutable upgrades Nomad 0.8 alloc runner state.
func upgradeOldAllocMutable(tx *boltdd.Tx, allocID string, oldBytes []byte) error {
	var oldMutable allocRunnerMutableState08
	err := codec.NewDecoderBytes(oldBytes, structs.MsgpackHandle).Decode(&oldMutable)
	if err != nil {
		return err
	}

	// Upgrade Deployment Status
	if err := putDeploymentStatusImpl(tx, allocID, oldMutable.DeploymentStatus); err != nil {
		return err
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Accept the drop: this error intentionally discards the task bucket; the task will restart fresh under 0.9+
2. If the task mattered, restore the pre-upgrade backup of client state and fix the 0.8 agent first
3. Stop nomad, back up, and clean the data_dir if migration errors cascade on many allocs
4. Confirm on disk (bolt browser) that the task bucket truly lacks 'simple-all' before suspecting a bug

// Go: detect legacy task buckets missing 'simple-all' before upgrading
c := bkt.Cursor()
found := false
for k, v := c.First(); k != nil; k, v = c.Next() {
    if v != nil && bytes.Equal(k, []byte("simple-all")) {
        found = true
    }
}
if !found {
    logger.Warn("task bucket has no simple-all entry; task state will be dropped and task restarted")
}

When it happens

Trigger: A task sub-bucket with no 'simple-all' entry (empty bucket, or all entries were deleted as unexpected keys) is processed during UpgradeAllocs.

Common situations: Task buckets created but never populated due to a 0.8 crash; state files tampered with or partially restored from backup; buckets left by interrupted previous upgrade attempts.

Related errors

• error deleting invalid task state for task %q: %v
• error deleting unexpected task bucket %q: %v
• error delting unexpected task key %q: %v
• failed to read dynamic plugin registry state: %v
• timed out while opening database, is another Nomad process a

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/d653d7207b2bcea5. Report an issue: GitHub.