thanos-io/thanos · error

open block

Error message

open block %v

What it means

Wraps tsdb.OpenBlock failing when the rewrite command opens the downloaded block to read its index and chunks. OpenBlock validates the block's index format, chunk encoding, and internal consistency; a failure means the block data on disk is corrupt, from an incompatible TSDB version, or incompletely downloaded.

Solutions

1. Re-run with a fresh tmpDir to rule out a partial download, then retry.
2. Verify block integrity with `thanos tools bucket verify` and check the block against its meta.json hash; restore the block from a backup if corrupted.
3. Upgrade (or downgrade) the thanos binary so its embedded TSDB version supports the block's index format.

Example fix

// before
thanos tools bucket rewrite --objstore.bucket=b ID
// after (fresh temp dir and current binary)
thanos tools bucket rewrite --objstore.bucket=b --tmp.dir=$(mktemp -d) ID

b, err := tsdb.OpenBlock(slog, dir, chunkPool, nil)
if err != nil {
    level.Error(logger).Log("msg", "corrupt block", "id", id, "err", err)
    return errors.Wrapf(err, "open block %v; consider bucket verify/restore", id)
}

When it happens

Trigger: Block index or chunks corrupted in object store or during download; block written by a newer Prometheus/Thanos version using unsupported index format; truncated files on disk due to disk pressure; not enough memory to load index.

Common situations: Upgrading thanos binary behind the block format produced by a newer Prometheus; damaged object after manual object-store manipulation; interrupted earlier download; index file zero-length after a failed upload previously accepted by a broken uploader.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• found chunks outside the block time range introduced by…
• / series have an average of %.3f out-of-order chunks: %.3f…
• found chunks non-completely outside the block time range…
• joined health-check error messages (AnyErr)
• open index file

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/715e419d6474c1b1. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/tools_bucket.go:1259

		ctx, cancel := context.WithCancel(context.Background())
		g.Add(func() error {
			chunkPool := chunkenc.NewPool()
			changeLog := compactv2.NewChangeLog(io.Discard)
			stubCounter := promauto.With(nil).NewCounter(prometheus.CounterOpts{})
			for _, id := range ids {
				// Delete series from block & modify.
				level.Info(logger).Log("msg", "downloading block", "source", id)
				if err := block.Download(ctx, logger, insBkt, id, filepath.Join(tbc.tmpDir, id.String())); err != nil {
					return errors.Wrapf(err, "download %v", id)
				}

				meta, err := metadata.ReadFromDir(filepath.Join(tbc.tmpDir, id.String()))
				if err != nil {
					return errors.Wrapf(err, "read meta of %v", id)
				}
				b, err := tsdb.OpenBlock(logutil.GoKitLogToSlog(logger), filepath.Join(tbc.tmpDir, id.String()), chunkPool, nil)
				if err != nil {
					return errors.Wrapf(err, "open block %v", id)
				}

				p := compactv2.NewProgressLogger(logger, int(b.Meta().Stats.NumSeries))
				newID := ulid.MustNew(ulid.Now(), rand.Reader)
				meta.ULID = newID
				meta.Thanos.Rewrites = append(meta.Thanos.Rewrites, metadata.Rewrite{
					Sources:          meta.Compaction.Sources,
					DeletionsApplied: deletions,
					RelabelsApplied:  relabels,
				})
				meta.Compaction.Sources = []ulid.ULID{newID}
				meta.Thanos.Source = metadata.BucketRewriteSource

				if err := os.MkdirAll(filepath.Join(tbc.tmpDir, newID.String()), os.ModePerm); err != nil {
					return err
				}

				if *provideChangeLog {

View on GitHub (pinned to 35b8b99117)