{"record":{"id":"9d708f66230606d4","repo":"thanos-io/thanos","slug":"postings","errorCode":null,"errorMessage":"postings","messagePattern":"postings","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/block/index.go","lineNumber":597,"sourceCode":"\tindexr tsdb.IndexReader, chunkr tsdb.ChunkReader,\n\tindexw tsdb.IndexWriter, chunkw tsdb.ChunkWriter,\n\tmeta *metadata.Meta,\n\tignoreChkFns []ignoreFnType,\n) error {\n\tsymbols := indexr.Symbols()\n\tfor symbols.Next() {\n\t\tif err := indexw.AddSymbol(symbols.At()); err != nil {\n\t\t\treturn errors.Wrap(err, \"add symbol\")\n\t\t}\n\t}\n\tif symbols.Err() != nil {\n\t\treturn errors.Wrap(symbols.Err(), \"next symbol\")\n\t}\n\n\tkey, value := index.AllPostingsKey()\n\tall, err := indexr.Postings(ctx, key, value)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"postings\")\n\t}\n\tall = indexr.SortedPostings(all)\n\n\t// We fully rebuild the postings list index from merged series.\n\tvar (\n\t\tpostings = index.NewMemPostings()\n\t\tvalues   = map[string]stringset{}\n\t\ti        = storage.SeriesRef(0)\n\t\tseries   = []seriesRepair{}\n\t)\n\n\tvar builder labels.ScratchBuilder\n\tvar chks []chunks.Meta\n\tfor all.Next() {\n\t\tid := all.At()\n\n\t\tif err := indexr.Series(id, &builder, &chks); err != nil {\n\t\t\treturn errors.Wrap(err, \"series\")","sourceCodeStart":579,"sourceCodeEnd":615,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/block/index.go#L579-L615","documentation":"This error wraps a failure from IndexReader.Postings(ctx, key, value) in block.Rewrite (pkg/block/index.go:597), where the code fetches the all-postings list using index.AllPostingsKey() to rebuild the postings index. It indicates the source index file's postings section cannot be read or decoded — a corrupt/truncated postings offset table — rather than a symbol or series problem.","triggerScenarios":"indexr.Postings(ctx, AllPostingsName, AllPostingsListValue) failing while reading the postings offset table for the special all-postings key during rewrite/Repair.","commonSituations":"Blocks damaged by crash during compaction; disk-level corruption; manually copied blocks missing parts of the index file; promtool repair on blocks produced by an older Prometheus index format.","solutions":["Restore the block from backup/replica or delete the corrupt block and let TSDB re-ingest or re-compact from upstream data.","Use promtool tsdb dump/analyze to confirm which index sections are readable and gauge data loss scope.","Check the block's index file size and checksums if the block was copied between nodes; re-copy if truncated.","Verify the reading Prometheus/promtool version is compatible with the block's index format version."],"exampleFix":"// before: rewrite proceeds and fails on unreadable postings\nerr := block.Rewrite(ctx, meta, src, dst, indexr, chunkr, nil)\n// after: pre-validate that all-postings are readable before rewrite\n_, err := indexr.Postings(ctx, index.AllPostingsKey())\nif err != nil { return fmt.Errorf(\"source index unreadable, restore block: %w\", err) }\nreturn block.Rewrite(ctx, meta, src, dst, indexr, chunkr, nil)","handlingStrategy":"validation","validationCode":"all, err := indexr.Postings(ctx, index.AllPostingsKey())\nif err != nil { return fmt.Errorf(\"source index postings unreadable; restore block: %w\", err) }\ndefer all.Close()","typeGuard":"func isPostingsError(err error) bool { return err != nil && strings.Contains(err.Error(), \"postings\") }","tryCatchPattern":"if err := block.Rewrite(ctx, meta, src, dst, indexr, chunkr, nil); err != nil {\n    if strings.Contains(err.Error(), \"postings\") {\n        // postings section corrupt: quarantine block, do not retry rewrite\n    }\n    return err\n}","preventionTips":["Pre-flight postings readability before launching a long repair","Verify index file integrity (size/checksum) after any block transfer","Keep Prometheus/promtool version compatible with block index format version","Quarantine corrupt blocks promptly so compaction skips them"],"tags":["tsdb","index","postings","corruption"],"backgroundTag":"file-read-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}