{"record":{"id":"a5fa26e1184b42da","repo":"thanos-io/thanos","slug":"read-series","errorCode":null,"errorMessage":"read series","messagePattern":"read series","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/block/index.go","lineNumber":279,"sourceCode":"\tif version >= 2 {\n\t\toffsetMultiplier = 16\n\t}\n\n\t// Per series.\n\tvar prevId storage.SeriesRef\n\tfor p.Next() {\n\t\tprevLset.CopyFrom(lset)\n\n\t\tid := p.At()\n\t\tif prevId != 0 {\n\t\t\t// Approximate size.\n\t\t\tseriesSize.Add(int64(id-prevId) * int64(offsetMultiplier))\n\t\t}\n\t\tprevId = id\n\t\tstats.TotalSeries++\n\n\t\tif err := r.Series(id, &builder, &chks); err != nil {\n\t\t\treturn stats, errors.Wrap(err, \"read series\")\n\t\t}\n\t\tlset = builder.Labels()\n\t\tif lset.IsEmpty() {\n\t\t\treturn stats, errors.Errorf(\"empty label set detected for series %d\", id)\n\t\t}\n\t\tif !prevLset.IsEmpty() && labels.Compare(prevLset, lset) >= 0 {\n\t\t\treturn stats, errors.Errorf(\"series %v out of order; previous %v\", lset, prevLset)\n\t\t}\n\t\tvar l0 *labels.Label\n\t\tlset.Range(func(l labels.Label) {\n\t\t\tif l0 != nil {\n\t\t\t\tif l.Name < l0.Name {\n\t\t\t\t\tstats.OutOfOrderLabels++\n\t\t\t\t\tlevel.Warn(logger).Log(\"msg\",\n\t\t\t\t\t\t\"out-of-order label set: known bug in Prometheus 2.8.0 and below\",\n\t\t\t\t\t\t\"labelset\", lset.String(),\n\t\t\t\t\t\t\"series\", fmt.Sprintf(\"%d\", id),\n\t\t\t\t\t)","sourceCodeStart":261,"sourceCodeEnd":297,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/block/index.go#L261-L297","documentation":"For each posting ID, GatherIndexHealthStats calls r.Series(id, ...) to decode a series' labels and chunks. If the series entry cannot be read/decoded from the index, the error is wrapped as \"read series\". This means the series section of the index references data that is missing or malformed.","triggerScenarios":"GatherIndexHealthStats iterating postings of a block whose series entries are truncated or misaligned (e.g. wrong offsetMultiplier handling for index format, damaged index file), or ctx cancellation.","commonSituations":"Index files produced by buggy or downgraded TSDB writers, hand-copied blocks missing parts of the index, storage bit-rot, corrupted uploads.","solutions":["Re-obtain the block from object storage or backup","Verify with VerifyIndex / tsdb tooling to pinpoint the bad series offset","Rebuild the block index via offline compaction of its chunks if the data is intact","Drop the corrupt block"],"exampleFix":"// before\nstats, err := block.GatherIndexHealthStats(ctx, bdir) // read series: invalid series ref\n// after\nresid, err := block.Repair(ctx, logger, filepath.Dir(bdir), id, meta.Thanos.Source)\nif err != nil { logger.Log(\"msg\", \"repair failed, dropping block\", \"err\", err) }","handlingStrategy":"try-catch","validationCode":"meta, err := metadata.ReadFromDir(bdir)\nif err != nil { return err }\nif meta.Thanos.Downsample.Resolution < 0 { return fmt.Errorf(\"invalid metadata in %s\", bdir) }","typeGuard":null,"tryCatchPattern":"stats, err := block.GatherIndexHealthStats(ctx, bdir)\nif err != nil && strings.Contains(err.Error(), \"read series\") {\n    return fmt.Errorf(\"series section corrupt in %s, rebuild via compaction: %w\", bdir, err)\n}","preventionTips":["Never hand-edit or partially copy block directories","Keep Prometheus/Thanos on versions without known index writer bugs","Verify blocks after every download","Rebuild indexes with offline compaction when chunk data is intact"],"tags":["corruption","index","series"],"backgroundTag":"file-read-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}