thanos-io/thanos · error

read meta file

Error message

read meta file

What it means

Repair reads the block's meta.json via metadata.ReadFromDir(bdir) and wraps any failure as "read meta file". The block directory must contain a valid meta.json; missing, malformed, or unwritable metadata makes repair impossible. This also catches blocks whose meta.json is corrupted or whose directory is not actually a TSDB block.

Solutions

1. Ensure the block directory contains a valid meta.json before calling Repair
2. Re-sync/re-download the block from object storage so meta.json is restored
3. Validate meta.json parses (metadata.ReadFromDir) and fix JSON syntax errors
4. Check filesystem permissions on the block directory

Example fix

// before
resid, err := block.Repair(ctx, logger, dir, id, source, fn) // read meta file: no such file or directory
// after
if _, err := os.Stat(filepath.Join(dir, id.String(), "meta.json")); os.IsNotExist(err) {
    return fmt.Errorf("block %s has no meta.json; re-sync it from object storage", id)
}
resid, err := block.Repair(ctx, logger, dir, id, source, fn)

meta, err := metadata.ReadFromDir(filepath.Join(dir, id.String()))
if err != nil {
    return fmt.Errorf("block %s unusable, meta.json missing or invalid: %w", id, err)
}

func blockDirHasMeta(dir string, id ulid.ULID) bool {
    _, err := os.Stat(filepath.Join(dir, id.String(), "meta.json"))
    return err == nil
}

resid, err := block.Repair(ctx, logger, dir, id, source, fn)
if err != nil && strings.Contains(err.Error(), "read meta file") {
    return fmt.Errorf("re-sync block %s from object storage: %w", id, err)
}

When it happens

Trigger: Calling Repair (or repairIndex via VerifyIndex repair paths) on a directory lacking meta.json, with invalid JSON in meta.json, or where reading the file fails due to permissions.

Common situations: Partially uploaded/downloaded blocks missing meta.json, manually moved block directories, objects storage sync tools that skip metadata files, permissions problems on local disks.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• postings offsets for
• critical error detected
• could not sync metas
• create meta fetcher
• sync before first pass of downsampling

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/5a6cd6a4c21d76e9. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/index.go:426

// Repair open the block with given id in dir and creates a new one with fixed data.
// It:
// - removes out of order duplicates
// - all "complete" outsiders (they will not accessed anyway)
// - removes all near "complete" outside chunks introduced by https://github.com/prometheus/tsdb/issues/347.
// Fixable inconsistencies are resolved in the new block.
// TODO(bplotka): https://github.com/thanos-io/thanos/issues/378.
func Repair(ctx context.Context, logger log.Logger, dir string, id ulid.ULID, source metadata.SourceType, ignoreChkFns ...ignoreFnType) (resid ulid.ULID, err error) {
	if len(ignoreChkFns) == 0 {
		return resid, errors.New("no ignore chunk function specified")
	}

	bdir := filepath.Join(dir, id.String())
	entropy := rand.New(rand.NewSource(time.Now().UnixNano()))
	resid = ulid.MustNew(ulid.Now(), entropy)

	meta, err := metadata.ReadFromDir(bdir)
	if err != nil {
		return resid, errors.Wrap(err, "read meta file")
	}
	if meta.Thanos.Downsample.Resolution > 0 {
		return resid, errors.New("cannot repair downsampled block")
	}

	b, err := tsdb.OpenBlock(logutil.GoKitLogToSlog(logger), bdir, nil, nil)
	if err != nil {
		return resid, errors.Wrap(err, "open block")
	}
	defer runutil.CloseWithErrCapture(&err, b, "repair block reader")

	indexr, err := b.Index()
	if err != nil {
		return resid, errors.Wrap(err, "open index")
	}
	defer runutil.CloseWithErrCapture(&err, indexr, "repair index reader")

	chunkr, err := b.Chunks()

View on GitHub (pinned to 35b8b99117)