thanos-io/thanos · error

chunk iter

Error message

chunk iter

What it means

During per-series chunk accumulation in write(), chksIter.Err() was non-nil after draining the series' chunk iterator. The error means enumerating a series' chunks (from the populated lazy set) failed mid-way — typically an error chunk from the deferred population of chunk data.

Solutions

1. Follow the wrapped cause to the failing chunk Ref; validate and repair/replace the corrupt source block.
2. Re-sync blocks from object storage to rule out truncated uploads (verify file sizes/checksums).
3. Retry compaction; if persistent, remove the bad block from the compaction set.
4. Check compactor host disk health (dmesg, SMART) for underlying IO errors.

// pre-check chunk files of the source block before building the set
if err := verifyChunkSegmentFiles(blockDir); err != nil {
    excludeFromCompaction(blockDir, err)
}

func chunkIterHealthy(it chunkIterable) bool { return it.Err() == nil }

if err := WriteSeries(ctx, set, symbols); err != nil {
    if isChunkCorruption(err) {
        markBlockForRepair(sourceBlockID)
    }
    return err
}

When it happens

Trigger: write() appends chunks from populatedSet's chksIter per series; a previous lazyPopulatableChunk.populate failure surfaces here as the iterator's error (e.g. unreadable chunk bytes), or the chunk list iterator itself failed.

Common situations: Corrupt or truncated source chunk files, partially uploaded blocks in object storage, disk read errors on the compactor node.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• cannot populate chunk
• symbols
• write chunks
• iterate chunk while re-encoding
• get series

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/86d39e501e3d93dc. Report an issue: GitHub.

Appendix: source

Thrown at pkg/compactv2/chunk_series_set.go:209

	// Iterate over all sorted chunk series.
	for populatedSet.Next() {
		select {
		case <-ctx.Done():
			return ctx.Err()
		default:
		}

		s := populatedSet.At()
		chksIter := s.Iterator(nil)
		chks = chks[:0]
		for chksIter.Next() {
			// We are not iterating in streaming way over chunk as it's more efficient to do bulk write for index and
			// chunk file purposes.
			chks = append(chks, chksIter.At())
		}

		if chksIter.Err() != nil {
			return errors.Wrap(chksIter.Err(), "chunk iter")
		}

		// Skip the series with all deleted chunks.
		if len(chks) == 0 {
			// All series will be ignored.
			p.SeriesProcessed()
			continue
		}

		if err := sWriter.WriteChunks(chks...); err != nil {
			return errors.Wrap(err, "write chunks")
		}
		if err := sWriter.AddSeries(ref, s.Labels(), chks...); err != nil {
			return errors.Wrap(err, "add series")
		}
		for _, chk := range chks {
			// ChunkPool is used by tsdb.OpenBlock BlockReader.
			if err := w.chunkPool.Put(chk.Chunk); err != nil {

View on GitHub (pinned to 35b8b99117)