{"record":{"id":"ffb06b2382afa3a7","repo":"hashicorp/nomad","slug":"failed-to-decode-log-entry-at-index-d-v","errorCode":null,"errorMessage":"failed to decode log entry at index %d: %v","messagePattern":"failed to decode log entry at index (.+?): (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"helper/raftutil/state.go","lineNumber":129,"sourceCode":"\tentries := make(chan interface{})\n\twarnings := make(chan error)\n\n\tgo func() {\n\t\tdefer store.Close()\n\t\tdefer close(entries)\n\t\tfor i := firstIdx; i <= lastIdx; i++ {\n\t\t\tvar e raft.Log\n\t\t\terr := store.GetLog(i, &e)\n\t\t\tif err != nil {\n\t\t\t\twarnings <- fmt.Errorf(\n\t\t\t\t\t\"failed to read log entry at index %d (firstIdx: %d, lastIdx: %d): %v\",\n\t\t\t\t\ti, firstIdx, lastIdx, err)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tentry, err := decode(&e)\n\t\t\tif err != nil {\n\t\t\t\twarnings <- fmt.Errorf(\n\t\t\t\t\t\"failed to decode log entry at index %d: %v\", i, err)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tentries <- entry\n\t\t}\n\t}()\n\n\treturn entries, warnings, nil\n}\n\ntype logMessage struct {\n\tLogType string\n\tTerm    uint64\n\tIndex   uint64\n\n\tCommandType           string      `json:\",omitempty\"`\n\tIgnoreUnknownTypeFlag bool        `json:\",omitempty\"`","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/helper/raftutil/state.go#L111-L147","documentation":"After GetLog succeeds, LogEntries decodes each raft.Log into a logMessage via decode(); if decode fails (e.g. a LogCommand entry with empty Data), the error is sent on the warnings channel and the entry is skipped. Note that msgpack body decode failures inside decode are swallowed (body replaced with \"FAILED TO DECODE DATA\"), so this warning fires only for structural failures like missing command data or an unknown log type with no mapping.","triggerScenarios":"A LogCommand entry whose Data is empty (decode returns 'command did not include data'); or entries written by an incompatible Nomad version whose Data does not conform to the expected layout when decoded.","commonSituations":"Reading logs produced by a much older/newer Nomad version with different message encoding; entries corrupted on disk so the command payload was truncated to zero length; inspecting logs written by third-party forks using unknown message types.","solutions":["Read the warnings channel and note which indexes failed; the rest of the log can still be inspected.","Check the Nomad version that wrote the data and use the matching version of the inspect tooling.","If entries are corrupt (empty command data), restore the raft store from a healthy backup or peer.","Report/inspect the raw raft.Log for those indexes with a custom reader if you need the bytes for forensic analysis.","Treat the server's log as suspect if many entries fail to decode; re-add the server to force log replication."],"exampleFix":"// before\nentry, err := decode(&e) // inside LogEntries; empty LogCommand data -> warning\n// after\n// caller-side: collect decode warnings and fall back to raw inspection\nfor w := range warningsCh {\n\tlog.Printf(\"skipping entry: %v\", w)\n}\n// keep raw entries for forensic dump:\nraw := &raft.Log{}\nif err := store.GetLog(i, raw); err == nil {\n\thexdump.Raw(raw.Data)\n}","handlingStrategy":"fallback","validationCode":"func readEntriesTolerantDecoding(p string) ([]*logMessage, []error, error) {\n\tentriesCh, warnCh, err := raftutil.LogEntries(p)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\tvar entries []*logMessage\n\tvar undecodable []error\n\tfor entriesCh != nil || warnCh != nil {\n\t\tselect {\n\t\tcase e, ok := <-entriesCh:\n\t\t\tif !ok { entriesCh = nil; continue }\n\t\t\tentries = append(entries, e.(*logMessage))\n\t\tcase w, ok := <-warnCh:\n\t\t\tif !ok { warnCh = nil; continue }\n\t\t\tundecodable = append(undecodable, w)\n\t\t}\n\t}\n\treturn entries, undecodable, nil\n}","typeGuard":"func isDecodeWarning(w error) bool {\n\treturn w != nil && strings.Contains(w.Error(), \"failed to decode log entry at index\")\n}","tryCatchPattern":"for w := range warnings {\n\tif isDecodeWarning(w) {\n\t\tlog.Printf(\"undecodable entry skipped (version mismatch?): %v\", w)\n\t\t// fall back to raw byte inspection for those indexes\n\t}\n}","preventionTips":["Use the same Nomad version to inspect logs as the server that wrote them.","Always drain the warnings channel to avoid goroutine deadlock.","Remember msgpack body errors are downgraded to \"FAILED TO DECODE DATA\" — only structural decode errors warn.","Check for empty/truncated command data as a sign of on-disk corruption.","Keep version-compatible tooling and backups for forensic reads."],"tags":["raft","decoding","msgpack","versioning"],"backgroundTag":"raft-log-decode-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}