hashicorp/nomad · error

first index mismatch: source=%d, destination=%d

Error message

first index mismatch: source=%d, destination=%d

What it means

Fires in verifyMigration when the source and destination raft logs disagree on FirstIndex: the WAL copy is missing or contains extra leading entries, i.e. the migration is not faithful and must not be trusted.

Source

Thrown at helper/raftutil/migrate.go:273

	}

	srcLast, err := src.LastIndex()
	if err != nil {
		return fmt.Errorf("failed to get source last index: %w", err)
	}

	dstFirst, err := dst.FirstIndex()
	if err != nil {
		return fmt.Errorf("failed to get destination first index: %w", err)
	}

	dstLast, err := dst.LastIndex()
	if err != nil {
		return fmt.Errorf("failed to get destination last index: %w", err)
	}

	if srcFirst != dstFirst {
		return fmt.Errorf("first index mismatch: source=%d, destination=%d", srcFirst, dstFirst)
	}

	if srcLast != dstLast {
		return fmt.Errorf("last index mismatch: source=%d, destination=%d", srcLast, dstLast)
	}

	// Verify stable store uint64 keys.
	uint64Keys := []string{"CurrentTerm", "LastVoteTerm"}
	for _, key := range uint64Keys {
		srcVal, err := src.GetUint64([]byte(key))
		if err != nil {
			return fmt.Errorf("failed to get source uint64 key %s: %w", key, err)
		}

		dstVal, err := dst.GetUint64([]byte(key))
		if err != nil {
			return fmt.Errorf("failed to get destination uint64 key %s: %w", key, err)
		}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Remove the WAL directory and re-run nomad server migrate from the intact BoltDB store
2. Do not start the server on a mismatched store; restore from the BoltDB backup instead
3. Report persistent mismatches as a bug with the raft store versions in use

When it happens

Trigger: Thrown at helper/raftutil/migrate.go:273 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/771eb027028e1262. Report an issue: GitHub.