thanos-io/thanos · error

add chunk load

Error message

add chunk load

What it means

Wraps chunkr.addLoad failure while scheduling chunk fetches for a series in blockSeriesClient.series. addLoad registers a chunk reference for batched loading and fails when the same chunk reference is added twice in one batch or the request exceeds configured limits.

Solutions

1. Check the bucket for overlapping/duplicate blocks via 'thanos tools bucket verify' and remove the offending block.
2. Inspect the wrapped root error; duplicate-ref errors usually indicate index metadata corruption.
3. Raise or review store request limits if limit-related, and narrow query scope.
4. Restart the affected store gateway to clear stale batch state after fixing the bucket.

// Detect overlapping blocks in a time-range query before fan-out
for i := 1; i < len(blocks); i++ {
	if blocks[i].MinTime < blocks[i-1].MaxTime && sameSource(blocks[i], blocks[i-1]) {
		log.Warn("overlapping blocks from same source; possible duplicate upload")
	}
}

if err := query(ctx); err != nil && strings.Contains(err.Error(), "add chunk load") {
	// duplicate chunk ref usually means overlapping/duplicate blocks
	logger.Error("duplicate chunk reference; run bucket verify", "err", err)
	return err
}

When it happens

Trigger: For each chunk meta of a matched series, b.chunkr.addLoad(meta.Ref, len(b.entries), j) errors: duplicate chunk ref within the same series batch, or maxSeriesCount/maxChunkCount style request accounting violations.

Common situations: Overlapping blocks in the bucket causing duplicate chunk references for the same series, corrupted chunk metadata (duplicate Refs) in the index, misconfigured per-request limits.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• preload series
• load chunks
• invalid argument: --min-time
• error while parsing config for request logging
• empty chunks for series

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/a6fe62010086ea14. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/bucket.go:1377

		seriesMatched++
		if b.seriesLimit > 0 && seriesMatched > b.seriesLimit {
			// Exit early if seriesLimit is set.
			b.hasMorePostings = false
			break
		}
		s := seriesEntry{lset: completeLabelset}
		if b.skipChunks {
			b.entries = append(b.entries, s)
			continue
		}

		// Schedule loading chunks.
		s.refs = make([]chunks.ChunkRef, 0, len(b.chkMetas))
		s.chks = make([]storepb.AggrChunk, 0, len(b.chkMetas))

		for j, meta := range b.chkMetas {
			if err := b.chunkr.addLoad(meta.Ref, len(b.entries), j); err != nil {
				return errors.Wrap(err, "add chunk load")
			}
			s.chks = append(s.chks, storepb.AggrChunk{
				MinTime: meta.MinTime,
				MaxTime: meta.MaxTime,
			})
			s.refs = append(s.refs, meta.Ref)
		}

		// Ensure sample limit through chunksLimiter if we return chunks.
		if err := b.chunksLimiter.Reserve(uint64(len(b.chkMetas))); err != nil {
			return httpgrpc.Errorf(int(codes.ResourceExhausted), "exceeded chunks limit: %s", err)
		}

		b.entries = append(b.entries, s)
	}

	if lazyExpandedPosting {
		// Apply series limit before fetching chunks, for actual series matched.

View on GitHub (pinned to 35b8b99117)