thanos-io/thanos · error

generate index header

Error message

generate index header

What it means

In the in-memory path of loading an index-header (no cache filename configured), WriteBinary is called to generate the binary index-header bytes directly from the bucket. This error wraps any failure of that call, meaning the index-header could not be fetched/generated — root cause lies within WriteBinary (bucket access, malformed index, size limit).

Solutions

1. Check the wrapped cause: fix object-store connectivity/credentials if it is a download error.
2. Verify the block (meta.json + index) exists and is fully uploaded in the bucket; re-upload if truncated.
3. Configure a disk cache (cache path) so headers are fetched once and reused.
4. Check bucket rate limits (throttling) and add retries/backoff.

Example fix

null

// verify the block index exists before generating the header
ok, err := bkt.Exists(ctx, path.Join(id.String(), meta.FilenameMeta))
if err != nil || !ok {
    return fmt.Errorf("block %s missing from bucket", id)
}

null

_, err := NewBinaryReader(ctx, logger, bkt, "", id, sampling, metrics)
if err != nil && strings.Contains(err.Error(), "generate index header") {
    if isRetryable(err) {
        return retryWithBackoff(ctx, 3, func() error {
            _, err = NewBinaryReader(ctx, logger, bkt, "", id, sampling, metrics)
            return err
        })
    }
    return err
}

When it happens

Trigger: NewBinaryReader with cacheFilename == "": WriteBinary fails due to object-store read failure, block index not found in bucket, checksum failure, or the 64GiB size-limit error.

Common situations: Store gateway without disk cache configured hitting bucket throttling/outage; missing block index in bucket (partial upload/deletion); wrong bucket config pointing at a bucket missing the block.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• write index header
• invalid magic number
• write index header
• create index header reader
• index header PostingsOffset

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/b2c484bfad0debb6. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/indexheader/binary_reader.go:590

		binfn := filepath.Join(dir, id.String(), block.IndexHeaderFilename)
		br, err := newFileBinaryReader(binfn, postingOffsetsInMemSampling, metrics)
		if err == nil {
			return br, nil
		}

		level.Debug(logger).Log("msg", "failed to read index-header from disk; recreating", "path", binfn, "err", err)

		start := time.Now()
		if _, err := WriteBinary(ctx, bkt, id, binfn, metrics.downloadDuration); err != nil {
			return nil, errors.Wrap(err, "write index header")
		}

		level.Debug(logger).Log("msg", "built index-header file", "path", binfn, "elapsed", time.Since(start))
		return newFileBinaryReader(binfn, postingOffsetsInMemSampling, metrics)
	} else {
		buf, err := WriteBinary(ctx, bkt, id, "", metrics.downloadDuration)
		if err != nil {
			return nil, errors.Wrap(err, "generate index header")
		}

		return newMemoryBinaryReader(buf, postingOffsetsInMemSampling, metrics)
	}
}

func newMemoryBinaryReader(buf []byte, postingOffsetsInMemSampling int, metrics *BinaryReaderMetrics) (bw *BinaryReader, err error) {
	r := &BinaryReader{
		b:                           realByteSlice(buf),
		c:                           nil,
		postings:                    map[string]*postingValueOffsets{},
		postingOffsetsInMemSampling: postingOffsetsInMemSampling,
		metrics:                     metrics,
	}

	if err := r.init(); err != nil {
		return nil, err
	}

View on GitHub (pinned to 35b8b99117)