thanos-io/thanos · critical

open chunks

Error message

open chunks

What it means

block.Repair wraps the error from b.Chunks(), which opens a reader over the block's chunks/ segment files after the index opened fine. Failure indicates the chunk segment files (chunks/000001, ...) are missing, truncated, or fail CRC/decoding checks. Wrapped as 'open chunks: <cause>'.

Solutions

1. Check chunks/ dir listing in the block dir: segment files must be contiguous and non-empty (use block.GetSegmentFiles).
2. Re-download the block from object storage, verifying file sizes/checksums, then retry Repair.
3. Check disk space and permissions on the node running Repair (ENOSPC/EACCES show up here).
4. If chunks are unrecoverable, delete the block and rebuild from source data.

Example fix

// before
segfiles := block.GetSegmentFiles(bdir)
_ = segfiles // assume segments are fine
block.RepairIssue347(ctx, logger, dir, id, source)
// after
segfiles := block.GetSegmentFiles(bdir)
if len(segfiles) == 0 {
    return fmt.Errorf("block %s has no chunk segments: re-download before repair", id)
}

bdir := filepath.Join(dir, id.String())
segfiles := block.GetSegmentFiles(bdir)
if len(segfiles) == 0 {
    return fmt.Errorf("block %s has no chunk segments", id)
}
for _, sf := range segfiles {
    if fi, err := os.Stat(sf); err != nil || fi.Size() == 0 {
        return fmt.Errorf("chunk segment %s missing or empty", sf)
    }
}

resid, err := block.RepairIssue347(ctx, logger, dir, id, source)
if err != nil && strings.Contains(err.Error(), "open chunks") {
    if rerr := redownloadBlock(ctx, bucket, dir, id); rerr != nil {
        return resid, err
    }
    resid, err = block.RepairIssue347(ctx, logger, dir, id, source)
}

When it happens

Trigger: Repair on a block with damaged or missing chunk segment files; segment files truncated by a partial upload/download; chunks dir removed or renamed; file permission or I/O errors when memory-mapping/reading segments.

Common situations: Blocks where a sync tool dropped some chunk segments; interrupted multipart upload to object storage; repairing a block restored from an incomplete backup; filesystem errors on the node running the repair job.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• open block
• open block
• chunk read
• iterate series
• add series

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/8b9df201d27a13bd. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/index.go:446

	if meta.Thanos.Downsample.Resolution > 0 {
		return resid, errors.New("cannot repair downsampled block")
	}

	b, err := tsdb.OpenBlock(logutil.GoKitLogToSlog(logger), bdir, nil, nil)
	if err != nil {
		return resid, errors.Wrap(err, "open block")
	}
	defer runutil.CloseWithErrCapture(&err, b, "repair block reader")

	indexr, err := b.Index()
	if err != nil {
		return resid, errors.Wrap(err, "open index")
	}
	defer runutil.CloseWithErrCapture(&err, indexr, "repair index reader")

	chunkr, err := b.Chunks()
	if err != nil {
		return resid, errors.Wrap(err, "open chunks")
	}
	defer runutil.CloseWithErrCapture(&err, chunkr, "repair chunk reader")

	resdir := filepath.Join(dir, resid.String())

	chunkw, err := chunks.NewWriter(filepath.Join(resdir, ChunksDirname))
	if err != nil {
		return resid, errors.Wrap(err, "open chunk writer")
	}
	defer runutil.CloseWithErrCapture(&err, chunkw, "repair chunk writer")

	indexw, err := index.NewWriter(context.TODO(), filepath.Join(resdir, IndexFilename))
	if err != nil {
		return resid, errors.Wrap(err, "open index writer")
	}
	defer runutil.CloseWithErrCapture(&err, indexw, "repair index writer")

	// TODO(fabxc): adapt so we properly handle the version once we update to an upstream

View on GitHub (pinned to 35b8b99117)