thanos-io/thanos · error

read series

Error message

read series

What it means

Wraps IndexReader.LoadSeriesForTime failure in blockSeriesClient.series. This step decodes each preloaded series' symbols and chunk metadata, filtering by the query time range (mint/maxt). Failure indicates corrupt or unreadable series data in the block index.

Solutions

1. Identify the failing block via the wrapped error and run 'thanos tools bucket verify'.
2. Delete or restore the corrupted block so the compactor re-uploads a healthy copy.
3. Check for storage-layer bit rot (object-store checksum features, disk health on store gateway cache dirs).
4. Confirm all components run Prometheus/Thanos versions with compatible index formats.

if err := query(ctx); err != nil && strings.Contains(err.Error(), "read series") {
	logger.Error("series decode failed; possible corrupt block", "err", err)
	// fall back to partial results or failover to another store instance
	return failoverQuery(ctx, q)
}

When it happens

Trigger: For each posting batch entry, indexr.LoadSeriesForTime(posting, &symbolizedLset, &chkMetas, skipChunks, mint, maxt) errors while decoding a series entry or its chunk metadata from the index.

Common situations: Bit-corrupted index files (bad disk/object store), blocks written by mismatched Prometheus index versions, memory pressure corrupting reader caches.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• input block index not valid
• get all postings
• label names
• metric label values
• read series

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/6413611977cc8b2f. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/bucket.go:1318

	b.indexr.reset(len(postingsBatch))
	if !b.skipChunks {
		b.chunkr.reset()
	}

	if err := b.indexr.PreloadSeries(b.ctx, postingsBatch, b.bytesLimiter, b.tenant); err != nil {
		return errors.Wrap(err, "preload series")
	}

	seriesMatched := 0
	b.entries = b.entries[:0]
OUTER:
	for i := range postingsBatch {
		if err := b.ctx.Err(); err != nil {
			return err
		}
		hasMatchedChunks, err := b.indexr.LoadSeriesForTime(postingsBatch[i], &b.symbolizedLset, &b.chkMetas, b.skipChunks, b.mint, b.maxt)
		if err != nil {
			return errors.Wrap(err, "read series")
		}
		// Skip getting series symbols if we know there is no matched chunks
		// and lazy expanded posting not enabled.
		if !lazyExpandedPosting && !hasMatchedChunks {
			continue
		}

		if err := b.indexr.LookupLabelsSymbols(b.ctx, b.symbolizedLset, b.b); err != nil {
			return errors.Wrap(err, "Lookup labels symbols")
		}
		b.lset = b.b.Labels()

		for _, matcher := range b.lazyPostings.matchers {
			val := b.lset.Get(matcher.Name)
			if !matcher.Matches(val) {
				// Series not matched means series we overfetched due to lazy posting expansion.
				seriesBytes := b.indexr.loadedSeries[postingsBatch[i]]
				b.lazyExpandedPostingSeriesOverfetchedSizeBytes.Add(float64(len(seriesBytes)))

View on GitHub (pinned to 35b8b99117)