thanos-io/thanos · error

load chunks

Error message

load chunks

What it means

Wraps chunkr.load failure in blockSeriesClient.series. load executes all scheduled chunk fetches, fetching chunk data from the object store via the block's chunk reader and applying bytes limits. This is the final and most data-heavy step of series serving, so failures usually reflect storage access or size-limit problems.

Solutions

1. Read the wrapped root error: for byte-limit issues, lower query time range or increase store request-size limits.
2. Verify chunk integrity with 'thanos tools bucket verify'; remove corrupted blocks and let them be re-uploaded.
3. Check object-store error rates/limits (throttling, timeouts) and retry transient failures.
4. Ensure query concurrency and chunk-pool settings match the store gateway's resources.

// Preflight: ensure query fits within bytes limit
estimated := len(matchers) * avgSeriesPerMatcher * avgChunksPerSeries * avgChunkSize
if estimated > maxRequestBytes {
	return fmt.Errorf("estimated payload %d exceeds limit %d", estimated, maxRequestBytes)
}

err := store.Series(ctx, hints, mint, maxt, matchers)
for attempt := 0; err != nil && strings.Contains(err.Error(), "load chunks") && attempt < 3; attempt++ {
	if !isTransient(err) {
		break
	}
	time.Sleep(backoffFor(attempt))
	err = store.Series(ctx, hints, mint, maxt, matchers)
}

When it happens

Trigger: skipChunks is false and b.chunkr.load(ctx, entries, loadAggregates, calculateChunkHash, bytesLimiter, tenant) errors: chunk fetch from object storage failed, chunk data is corrupt/decode failed, or the bytes limiter rejected the request size.

Common situations: Object-store throttling or timeouts on wide queries, corrupted chunk files in the bucket, queries exceeding configured per-request bytes limits, blocks deleted mid-query.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• new bucket block
• add chunk load
• create meta fetcher
• create syncer
• compaction

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/98e949eac43a9d4f. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/bucket.go:1403

		// Ensure sample limit through chunksLimiter if we return chunks.
		if err := b.chunksLimiter.Reserve(uint64(len(b.chkMetas))); err != nil {
			return httpgrpc.Errorf(int(codes.ResourceExhausted), "exceeded chunks limit: %s", err)
		}

		b.entries = append(b.entries, s)
	}

	if lazyExpandedPosting {
		// Apply series limit before fetching chunks, for actual series matched.
		if err := b.seriesLimiter.Reserve(uint64(seriesMatched)); err != nil {
			return httpgrpc.Errorf(int(codes.ResourceExhausted), "exceeded series limit: %s", err)
		}
	}

	if !b.skipChunks {
		if err := b.chunkr.load(b.ctx, b.entries, b.loadAggregates, b.calculateChunkHash, b.bytesLimiter, b.tenant); err != nil {
			return errors.Wrap(err, "load chunks")
		}
	}

	return nil
}

func populateChunk(out *storepb.AggrChunk, in chunkenc.Chunk, aggrs []storepb.Aggr, save func([]byte) ([]byte, error), calculateChecksum bool) error {
	hasher := hashPool.Get().(hash.Hash64)
	defer hashPool.Put(hasher)

	if in.Encoding() == chunkenc.EncXOR || in.Encoding() == chunkenc.EncHistogram || in.Encoding() == chunkenc.EncFloatHistogram {
		b, err := save(in.Bytes())
		if err != nil {
			return err
		}
		out.Raw = &storepb.Chunk{
			Data: b,
			Type: chunkToStoreEncoding(in.Encoding()),

View on GitHub (pinned to 35b8b99117)