thanos-io/thanos · error

chunk read

Error message

chunk read

What it means

This error wraps a failure from ChunkReader.ChunkOrIterable(c) in block.Rewrite (pkg/block/index.go:624) while loading each chunk referenced by a series. It means a chunk referenced from the index cannot be read from the chunk segment files — the chunk reference is invalid, the segment file is missing/truncated, or chunk data fails integrity checks. This is the most common corruption symptom because chunk data lives in separate segments/ files.

Solutions

1. Restore the affected block's segments/ files from backup or another replica; the index may be fine while chunk data is missing.
2. Run the repair with ignoreChkFns that filter the unreadable chunk series so sanitizeChunkSequence drops them instead of failing the rewrite.
3. Verify all segments/ files are present and complete (sizes match, no partial transfers) after any block copy; re-copy if truncated.
4. Check dmesg/storage health for disk errors; move TSDB data to healthy storage and re-sync the block.
5. As last resort, quarantine the block and re-ingest or backfill the affected time range from remote storage.

Example fix

// before: rewrite aborts on any unreadable chunk
chks[i].Chunk, _, err = chunkr.ChunkOrIterable(c)
if err != nil { return errors.Wrap(err, "chunk read") }
// after: repair path ignores known-bad chunk ranges via ignoreChkFns
ignore := func(m chunks.Meta, _ int64, _ int64) bool {
    return m.MinTime >= badRangeStart && m.MaxTime <= badRangeEnd
}
err := block.Rewrite(ctx, meta, src, dst, indexr, chunkr, []block.IgnoreChunkFunc{ignore})

entries, err := os.ReadDir(filepath.Join(blockDir, "chunks"))
if err != nil || len(entries) == 0 { return fmt.Errorf("chunk segments missing for block %s", blockDir) }

func isChunkReadError(err error) bool { return err != nil && strings.Contains(err.Error(), "chunk read") }

if err := block.Rewrite(ctx, meta, src, dst, indexr, chunkr, nil); err != nil {
    if strings.Contains(err.Error(), "chunk read") {
        // fall back to rewrite with ignoreChkFns dropping unreadable chunk ranges
    }
    return err
}

When it happens

Trigger: chunkr.ChunkOrIterable(c) returning an error when the chunk ref (segment index + offset) points outside an existing segment, the segments/ file is truncated, or the chunk fails XOR chunk decoding/integrity validation.

Common situations: Missing or truncated segments/NNN files after interrupted copy or disk full during head compaction; blocks moved between nodes without all segment files; storage-level corruption (bad sectors); repairing blocks where ignoreChkFns should have been supplied to drop known-bad chunks.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• open chunks
• next symbol
• repair failed for block
• open block
• found chunks non-completely outside the block time range…

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/a9bd69a39e131acd. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/index.go:624

		series   = []seriesRepair{}
	)

	var builder labels.ScratchBuilder
	var chks []chunks.Meta
	for all.Next() {
		id := all.At()

		if err := indexr.Series(id, &builder, &chks); err != nil {
			return errors.Wrap(err, "series")
		}
		// Make sure labels are in sorted order.
		builder.Sort()

		for i, c := range chks {
			// Ignore iterable as it should be nil.
			chks[i].Chunk, _, err = chunkr.ChunkOrIterable(c)
			if err != nil {
				return errors.Wrap(err, "chunk read")
			}
		}

		chks, err := sanitizeChunkSequence(chks, meta.MinTime, meta.MaxTime, ignoreChkFns)
		if err != nil {
			return err
		}

		if len(chks) == 0 {
			continue
		}

		series = append(series, seriesRepair{
			lset: builder.Labels(),
			chks: chks,
		})
	}

View on GitHub (pinned to 35b8b99117)