thanos-io/thanos · critical

open index file

Error message

open index file

What it means

GatherIndexHealthStats wraps failures from index.NewFileReader with the message 'open index file'. This means the block's index file could not be opened/parsed as a TSDB index (missing file, truncated download, corrupt magic/TOC, wrong format version), so the index health walk cannot even begin.

Solutions

1. Check the file exists and is readable at fn (ls/stat the path) and that you are pointing at the block directory's 'index' file.
2. Re-download or re-upload the block to object storage; check sync/upload logs for truncation.
3. Restore the block from a backup or re-upload from source Prometheus data; delete the corrupted block so compactor can replace it.
4. Check disk space, permissions, and mount health on the local cache directory if blocks are synced locally.

Example fix

// before: path may not contain index
stats, err := block.GatherIndexHealthStats(ctx, logger, filepath.Join(dir, id.String()), minTime, maxTime)
// after: validate path first
indexFn := filepath.Join(dir, id.String(), block.IndexFilename)
if _, err := os.Stat(indexFn); err != nil {
    return errors.Wrapf(err, "index file missing for block %s", id)
}
stats, err := block.GatherIndexHealthStats(ctx, logger, indexFn, minTime, maxTime)

indexFn := filepath.Join(dir, id.String(), "index")
fi, err := os.Stat(indexFn)
if err != nil { return errors.Wrapf(err, "index file missing: %s", indexFn) }
if fi.Size() == 0 { return errors.Errorf("index file is empty: %s", indexFn) }

func indexFileReadable(path string) bool {
    fi, err := os.Stat(path)
    return err == nil && !fi.IsDir() && fi.Size() > 0
}

stats, err := block.GatherIndexHealthStats(ctx, logger, indexFn, minTime, maxTime)
if err != nil {
    if strings.Contains(err.Error(), "open index file") {
        return reDownloadBlock(ctx, dir, id)
    }
    return err
}

When it happens

Trigger: Calling GatherIndexHealthStats, VerifyIndex, or processDownsampling (which all call it) with an fn path where the index file is absent, unreadable, partially uploaded to object storage, or not a valid TSDB index file; index.NewFileReader fails and the error is wrapped with 'open index file'.

Common situations: Block directory missing the 'index' file because upload was interrupted; corrupt or partially synced object storage download; permissions preventing read; the file path passed points to the wrong directory; disk/network errors during local block sync.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• open block
• found chunks outside the block time range introduced by…
• / series have an average of %.3f out-of-order chunks: %.3f…
• found chunks non-completely outside the block time range…
• joined health-check error messages (AnyErr)

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/0492b611d2a65ab0. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/index.go:221

		n.max = v
	}
}

func (n *minMaxSumInt64) Avg() int64 {
	if n.cnt == 0 {
		return 0
	}
	return n.sum / n.cnt
}

// GatherIndexHealthStats returns useful counters as well as outsider chunks (chunks outside of block time range) that
// helps to assess index health.
// It considers https://github.com/prometheus/tsdb/issues/347 as something that Thanos can handle.
// See HealthStats.Issue347OutsideChunks for details.
func GatherIndexHealthStats(ctx context.Context, logger log.Logger, fn string, minTime, maxTime int64) (stats HealthStats, err error) {
	r, err := index.NewFileReader(fn, index.DecodePostingsRaw)
	if err != nil {
		return stats, errors.Wrap(err, "open index file")
	}
	defer runutil.CloseWithErrCapture(&err, r, "gather index issue file reader")

	key, value := index.AllPostingsKey()
	p, err := r.Postings(ctx, key, value)
	if err != nil {
		return stats, errors.Wrap(err, "get all postings")
	}
	var (
		lset     labels.Labels
		prevLset labels.Labels
		builder  labels.ScratchBuilder

		chks []chunks.Meta

		seriesLifeDuration                          = newMinMaxSumInt64()
		seriesLifeDurationWithoutSingleSampleSeries = newMinMaxSumInt64()
		seriesChunks                                = newMinMaxSumInt64()

View on GitHub (pinned to 35b8b99117)