hashicorp/nomad · error

missing logs [%v, %v]

Error message

missing logs [%v, %v]

What it means

ApplyNext returns this error when the next index to apply is below the store's first available log index, meaning the tail of the log has been compacted/truncated and the requested entries are gone. The missing index range [nextIdx, logFirstIdx-1] is reported.

Source

Thrown at helper/raftutil/fsm.go:128

}

func (f *FSMHelper) ApplyNext() (index uint64, term uint64, err error) {
	if f.nextIdx == 1 {
		// check snapshots first
		index, term, err := f.restoreFromSnapshot()
		if err != nil {
			return 0, 0, err
		}

		if index != 0 {
			f.nextIdx = index + 1
			return index, term, nil
		}
	}

	if f.nextIdx < f.logFirstIdx {
		return 0, 0, fmt.Errorf("missing logs [%v, %v]", f.nextIdx, f.logFirstIdx-1)
	}

	if f.nextIdx > f.logLastIdx {
		return 0, 0, ErrNoMoreLogs
	}

	var e raft.Log
	err = f.store.GetLog(f.nextIdx, &e)
	if err != nil {
		return 0, 0, fmt.Errorf("failed to read log entry at index %d: %v", f.nextIdx, err)
	}

	defer func() {
		r := recover()
		if r != nil && strings.HasPrefix(fmt.Sprint(r), "failed to apply request") {
			// Enterprise specific log entries will fail to load in OSS repository with "failed to apply request."
			// If not relevant to investigation, we can ignore them and simply worn.
			f.logger.Warn("failed to apply log; loading Enterprise data-dir in OSS binary?", "index", e.Index)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Start replay from the store's first index (RaftStateInfo's firstIdx) instead of an old cursor
2. Restore state from a snapshot at or before the desired index, then replay only subsequent logs
3. If the gap is unexpected, investigate compaction settings or store truncation

Example fix

// before
f, _ := raftutil.NewFSM(dir)
f.Seek(100) // may be compacted
// after
firstIdx, lastIdx, _ := raftutil.RaftStateInfo(storePath) // via helper
start := max(desiredIdx, firstIdx)
f.Seek(start)

firstIdx, _, err := storeFirstIndex(storePath) // via RaftStateInfo
if err != nil {
    return err
}
if replayStartIdx < firstIdx {
    replayStartIdx = firstIdx // skip compacted prefix
}

idx, term, err := f.ApplyNext()
if err != nil && strings.HasPrefix(err.Error(), "missing logs") {
    return fmt.Errorf("log %d is compacted; restore from snapshot then replay from first available index: %w", f.NextIndex(), err)
}

When it happens

Trigger: Calling ApplyNext (or ApplyUntil/ApplyAll which loop through it) with a starting index earlier than the store's first retained log, e.g. after snapshot compaction removed old entries.

Common situations: Trying to replay from an old index after Raft compaction; resuming an interrupted replay whose cursor predates the compacted prefix; pointing tooling at a store whose early logs were truncated.

Related errors

• no more logs
• failed to find raft store in %v: %v
• failed to open raft store %v: %v
• failed to open snapshot dir: %v
• failed to read log entry at index %d: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/b9408b45a3bdd7c0. Report an issue: GitHub.