{"record":{"id":"56b380ec8e2fdbb9","repo":"thanos-io/thanos","slug":"walk-postings","errorCode":null,"errorMessage":"walk postings","messagePattern":"walk postings","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/block/index.go","lineNumber":377,"sourceCode":"\t\t\tooo++\n\t\t}\n\t\tif ooo > 0 {\n\t\t\tstats.OutOfOrderSeries++\n\t\t\tstats.OutOfOrderChunks += ooo\n\t\t\tlevel.Debug(logger).Log(\"msg\", \"found out of order series\", \"labels\", lset)\n\t\t}\n\n\t\tseriesChunks.Add(int64(len(chks)))\n\t\tseriesLifeDuration.Add(seriesLifeTimeMs)\n\n\t\tif seriesLifeTimeMs == 0 {\n\t\t\tstats.SingleSampleSeries++\n\t\t} else {\n\t\t\tseriesLifeDurationWithoutSingleSampleSeries.Add(seriesLifeTimeMs)\n\t\t}\n\t}\n\tif err := p.Err(); err != nil {\n\t\treturn stats, errors.Wrap(err, \"walk postings\")\n\t}\n\n\tstats.SeriesMaxLifeDuration = time.Duration(seriesLifeDuration.max) * time.Millisecond\n\tstats.SeriesAvgLifeDuration = time.Duration(seriesLifeDuration.Avg()) * time.Millisecond\n\tstats.SeriesMinLifeDuration = time.Duration(seriesLifeDuration.min) * time.Millisecond\n\n\tstats.SeriesMaxLifeDurationWithoutSingleSampleSeries = time.Duration(seriesLifeDurationWithoutSingleSampleSeries.max) * time.Millisecond\n\tstats.SeriesAvgLifeDurationWithoutSingleSampleSeries = time.Duration(seriesLifeDurationWithoutSingleSampleSeries.Avg()) * time.Millisecond\n\tstats.SeriesMinLifeDurationWithoutSingleSampleSeries = time.Duration(seriesLifeDurationWithoutSingleSampleSeries.min) * time.Millisecond\n\n\tstats.SeriesMaxChunks = seriesChunks.max\n\tstats.SeriesAvgChunks = seriesChunks.Avg()\n\tstats.SeriesMinChunks = seriesChunks.min\n\n\tstats.ChunkMaxSize = chunkSize.max\n\tstats.ChunkAvgSize = chunkSize.Avg()\n\tstats.ChunkMinSize = chunkSize.min\n","sourceCodeStart":359,"sourceCodeEnd":395,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/block/index.go#L359-L395","documentation":"After walking all postings, GatherIndexHealthStats checks p.Err() and wraps any deferred postings-iteration error as \"walk postings\". Postings iterators surface corruption asynchronously (checksum/decoding errors mid-iteration), so this catches failures that occur while streaming series rather than at open time.","triggerScenarios":"The postings iterator hits a decoding or checksum failure partway through walking all series in GatherIndexHealthStats, or the context is cancelled during iteration.","commonSituations":"Bit-rot in index files, truncated block uploads that pass header checks, cancelled long verification runs over many blocks.","solutions":["Re-download or restore the block from source storage","Run VerifyIndex to confirm corruption location","Extend context timeouts for large verification sweeps","Remove the unreadable block"],"exampleFix":"// before\nstats, err := block.GatherIndexHealthStats(ctx, ctxBugCtx) // walk postings: context canceled\n// after\ncctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)\ndefer cancel()\nstats, err := block.GatherIndexHealthStats(cctx, bdir)","handlingStrategy":"try-catch","validationCode":"idxPath := filepath.Join(bdir, \"index\")\nfi, err := os.Stat(idxPath)\nif err != nil || fi.Size() == 0 { return fmt.Errorf(\"index missing/empty in %s\", bdir) }","typeGuard":null,"tryCatchPattern":"stats, err := block.GatherIndexHealthStats(ctx, bdir)\nif err != nil && strings.Contains(err.Error(), \"walk postings\") {\n    if errors.Is(ctx.Err(), context.DeadlineExceeded) { return retryLater }\n    return fmt.Errorf(\"block %s postings corrupt: %w\", bdir, err)\n}","preventionTips":["Distinguish context cancellation from corruption when handling this error","Retry transient cancellation; restore data for real decode errors","Use per-block timeouts in batch verification","Run checksum verification at the storage layer"],"tags":["corruption","index","iteration"],"backgroundTag":"file-read-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}