thanos-io/thanos · error

input block index not valid

Error message

input block index not valid

What it means

After downloading, processDownsampling calls block.VerifyIndex and, unless acceptMalformedIndex is set, wraps failures as "input block index not valid". The downloaded block's index file fails Thanos's health/consistency verification (index-header mismatches, out-of-range postings, corrupted file).

Solutions

1. Re-upload or regenerate the corrupt block from a healthy source (delete the bad block so it can be re-created from raw data).
2. Run with --accept-malformed-index if you have assessed the index issue and accept the risk (downsampling may produce partial results).
3. Verify minTime/maxTime in meta.json match the index; check for known Prometheus index-corruption issues for the version that wrote the block.

Example fix

// before
thanos downsample --data-dir=/data ...
// after (after confirming corruption is tolerated)
thanos downsample --data-dir=/data --accept-malformed-index ...

stats, err := block.GatherIndexHealthStats(ctx, logger, filepath.Join(bdir, block.IndexFilename), meta.MinTime, meta.MaxTime)
if err != nil || stats.AnyErr() != nil {
    return fmt.Errorf("source block %s index unhealthy, skipping", meta.ULID)
}

When it happens

Trigger: block.VerifyIndex(ctx, logger, bdir/index, m.MinTime, m.MaxTime) returns error and the --accept-malformed-index flag is false: corrupted or truncated index in the object store, index-header labels/resolution mismatch, malformed index produced by an old buggy Prometheus/Thanos.

Common situations: Blocks uploaded before a failed/interrupted upload (missing parts); known historic TSDB index bugs; blocks whose meta minTime/maxTime disagree with the index contents; restoring from a broken backup.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• output block index not valid
• get all postings
• label names
• metric label values
• read series

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/dff343ab192578e3. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/downsample.go:367

	m *metadata.Meta,
	dir string,
	resolution int64,
	hashFunc metadata.HashFunc,
	metrics *DownsampleMetrics,
	acceptMalformedIndex bool,
	blockFilesConcurrency int,
) error {
	begin := time.Now()
	bdir := filepath.Join(dir, m.ULID.String())

	err := block.Download(ctx, logger, bkt, m.ULID, bdir, objstore.WithFetchConcurrency(blockFilesConcurrency))
	if err != nil {
		return compact.NewRetryError(errors.Wrapf(err, "download block %s", m.ULID))
	}
	level.Info(logger).Log("msg", "downloaded block", "id", m.ULID, "duration", time.Since(begin), "duration_ms", time.Since(begin).Milliseconds())

	if err := block.VerifyIndex(ctx, logger, filepath.Join(bdir, block.IndexFilename), m.MinTime, m.MaxTime); err != nil && !acceptMalformedIndex {
		return errors.Wrap(err, "input block index not valid")
	}

	begin = time.Now()

	var pool chunkenc.Pool
	if m.Thanos.Downsample.Resolution == 0 {
		pool = chunkenc.NewPool()
	} else {
		pool = downsample.NewPool()
	}

	b, err := tsdb.OpenBlock(logutil.GoKitLogToSlog(logger), bdir, pool, nil)
	if err != nil {
		return errors.Wrapf(err, "open block %s", m.ULID)
	}
	defer runutil.CloseWithLogOnErr(log.With(logger, "outcome", "potential left mmap file handlers left"), b, "tsdb reader")

	id, err := downsample.Downsample(ctx, logger, m, b, dir, resolution)

View on GitHub (pinned to 35b8b99117)