thanos-io/thanos · error

label names for block

Error message

label names for block %s

What it means

While serving LabelNames, the store gateway opens a per-block index header reader and calls LabelNames() on the block's index. A failure wrapped with the block ULID means reading label names from that block's index failed — typically corrupt/truncated index-header files or a block being removed mid-request.

Solutions

1. Identify the block ULID from the message; verify its index files in the bucket (thanos tools bucket verify).
2. Delete the local cached index-header so the store gateway re-downloads it, then restart/reload.
3. Check for blocks pending deletion markers (deletion-mark.json) and sync store gateway metadata.
4. Re-upload/restore the block from a healthy source if the object is genuinely corrupt.

if strings.Contains(err.Error(), blkID) {
    // exclude this block and retry LabelNames without it
}

When it happens

Trigger: indexHeaderReader.LabelNames() errors because the .index file or its sidecar index-header is missing/corrupt, checksum fails, or the block was deleted by compaction while the reader was open.

Common situations: Interruption while downloading/offloading blocks leaving truncated index files; bucket lifecycle policies deleting blocks under an active store gateway; corrupted object storage uploads; stale index-header after block replacement.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• input block index not valid
• get all postings
• label names
• metric label values
• read series

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/00985d61a76b6b64. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/bucket.go:1929

		indexr := b.indexReader(blockLogger)

		g.Go(func() error {
			span, newCtx := tracing.StartSpan(gctx, "bucket_store_block_label_names", tracing.Tags{
				"block.id":         b.meta.ULID,
				"block.mint":       b.meta.MinTime,
				"block.maxt":       b.meta.MaxTime,
				"block.resolution": b.meta.Thanos.Downsample.Resolution,
			})
			defer span.Finish()
			defer runutil.CloseWithLogOnErr(blockLogger, indexr, "label names")

			var result []string
			if len(reqSeriesMatchersNoExtLabels) == 0 {
				// Do it via index reader to have pending reader registered correctly.
				// LabelNames are already sorted.
				res, err := indexr.block.indexHeaderReader.LabelNames()
				if err != nil {
					return errors.Wrapf(err, "label names for block %s", b.meta.ULID)
				}

				// Add  a set for the external labels as well.
				// We're not adding them directly to refs because there could be duplicates.
				// b.extLset is already sorted by label name, no need to sort it again.
				extRes := make([]string, 0, b.extLset.Len())
				b.extLset.Range(func(l labels.Label) {
					if _, ok := extLsetToRemove[l.Name]; !ok {
						extRes = append(extRes, l.Name)
					}
				})

				result = strutil.MergeSlices(int(req.Limit), res, extRes)
			} else {
				seriesReq := &storepb.SeriesRequest{
					MinTime:              req.Start,
					MaxTime:              req.End,
					SkipChunks:           true,

View on GitHub (pinned to 35b8b99117)