{"record":{"id":"b9408b45a3bdd7c0","repo":"hashicorp/nomad","slug":"missing-logs-v-v","errorCode":null,"errorMessage":"missing logs [%v, %v]","messagePattern":"missing logs \\[(.+?), (.+?)\\]","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"helper/raftutil/fsm.go","lineNumber":128,"sourceCode":"\n}\n\nfunc (f *FSMHelper) ApplyNext() (index uint64, term uint64, err error) {\n\tif f.nextIdx == 1 {\n\t\t// check snapshots first\n\t\tindex, term, err := f.restoreFromSnapshot()\n\t\tif err != nil {\n\t\t\treturn 0, 0, err\n\t\t}\n\n\t\tif index != 0 {\n\t\t\tf.nextIdx = index + 1\n\t\t\treturn index, term, nil\n\t\t}\n\t}\n\n\tif f.nextIdx < f.logFirstIdx {\n\t\treturn 0, 0, fmt.Errorf(\"missing logs [%v, %v]\", f.nextIdx, f.logFirstIdx-1)\n\t}\n\n\tif f.nextIdx > f.logLastIdx {\n\t\treturn 0, 0, ErrNoMoreLogs\n\t}\n\n\tvar e raft.Log\n\terr = f.store.GetLog(f.nextIdx, &e)\n\tif err != nil {\n\t\treturn 0, 0, fmt.Errorf(\"failed to read log entry at index %d: %v\", f.nextIdx, err)\n\t}\n\n\tdefer func() {\n\t\tr := recover()\n\t\tif r != nil && strings.HasPrefix(fmt.Sprint(r), \"failed to apply request\") {\n\t\t\t// Enterprise specific log entries will fail to load in OSS repository with \"failed to apply request.\"\n\t\t\t// If not relevant to investigation, we can ignore them and simply worn.\n\t\t\tf.logger.Warn(\"failed to apply log; loading Enterprise data-dir in OSS binary?\", \"index\", e.Index)","sourceCodeStart":110,"sourceCodeEnd":146,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/helper/raftutil/fsm.go#L110-L146","documentation":"ApplyNext returns this error when the next index to apply is below the store's first available log index, meaning the tail of the log has been compacted/truncated and the requested entries are gone. The missing index range [nextIdx, logFirstIdx-1] is reported.","triggerScenarios":"Calling ApplyNext (or ApplyUntil/ApplyAll which loop through it) with a starting index earlier than the store's first retained log, e.g. after snapshot compaction removed old entries.","commonSituations":"Trying to replay from an old index after Raft compaction; resuming an interrupted replay whose cursor predates the compacted prefix; pointing tooling at a store whose early logs were truncated.","solutions":["Start replay from the store's first index (RaftStateInfo's firstIdx) instead of an old cursor","Restore state from a snapshot at or before the desired index, then replay only subsequent logs","If the gap is unexpected, investigate compaction settings or store truncation"],"exampleFix":"// before\nf, _ := raftutil.NewFSM(dir)\nf.Seek(100) // may be compacted\n// after\nfirstIdx, lastIdx, _ := raftutil.RaftStateInfo(storePath) // via helper\nstart := max(desiredIdx, firstIdx)\nf.Seek(start)","handlingStrategy":"validation","validationCode":"firstIdx, _, err := storeFirstIndex(storePath) // via RaftStateInfo\nif err != nil {\n    return err\n}\nif replayStartIdx < firstIdx {\n    replayStartIdx = firstIdx // skip compacted prefix\n}","typeGuard":null,"tryCatchPattern":"idx, term, err := f.ApplyNext()\nif err != nil && strings.HasPrefix(err.Error(), \"missing logs\") {\n    return fmt.Errorf(\"log %d is compacted; restore from snapshot then replay from first available index: %w\", f.NextIndex(), err)\n}","preventionTips":["Anchor replay cursors to the store's first index rather than stale saved positions","Restore from a snapshot when replaying from an index older than the compaction point","Persist replay cursors atomically so interruptions resume at valid indexes","Account for Raft compaction when designing resumable replay tooling"],"tags":["raft","compaction","log-gap","go"],"backgroundTag":"raft-log-compacted","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}