{"record":{"id":"3f153b7cbabf2c4b","repo":"thanos-io/thanos","slug":"index-header-label-values-for-block-s","errorCode":null,"errorMessage":"index header label values for block %s","messagePattern":"index header label values for block (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/store/bucket.go","lineNumber":2155,"sourceCode":"\t\t\treqSeriesMatchersNoExtLabels = append(reqSeriesMatchersNoExtLabels, m)\n\t\t}\n\n\t\tsortedReqSeriesMatchersNoExtLabels := newSortedMatchers(reqSeriesMatchersNoExtLabels)\n\n\t\tresHints.AddQueriedBlock(b.meta.ULID)\n\n\t\tblockLogger := log.With(logger, \"block\", b.meta.ULID)\n\t\tindexr := b.indexReader(blockLogger)\n\n\t\tg.Go(func() error {\n\t\t\tdefer runutil.CloseWithLogOnErr(blockLogger, indexr, \"label values\")\n\n\t\t\tvar result []string\n\t\t\tif len(reqSeriesMatchersNoExtLabels) == 0 {\n\t\t\t\t// Do it via index reader to have pending reader registered correctly.\n\t\t\t\tres, err := indexr.block.indexHeaderReader.LabelValues(req.Label)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn errors.Wrapf(err, \"index header label values for block %s\", b.meta.ULID)\n\t\t\t\t}\n\n\t\t\t\t// Add the external label value as well.\n\t\t\t\tif extLabelValue := b.extLset.Get(req.Label); extLabelValue != \"\" {\n\t\t\t\t\tres = strutil.MergeSlices(int(req.Limit), res, []string{extLabelValue})\n\t\t\t\t}\n\t\t\t\tresult = res\n\t\t\t} else {\n\t\t\t\tseriesReq := &storepb.SeriesRequest{\n\t\t\t\t\tMinTime:              req.Start,\n\t\t\t\t\tMaxTime:              req.End,\n\t\t\t\t\tSkipChunks:           true,\n\t\t\t\t\tWithoutReplicaLabels: req.WithoutReplicaLabels,\n\t\t\t\t}\n\t\t\t\tblockClient := newBlockSeriesClient(\n\t\t\t\t\tgctx,\n\t\t\t\t\tblockLogger,\n\t\t\t\t\tb,","sourceCodeStart":2137,"sourceCodeEnd":2173,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/store/bucket.go#L2137-L2173","documentation":"For each queried block, BucketStore reads label values from the block's in-memory index header (indexHeaderReader.LabelValues). A failure reading/decoding the cached index-header file for that block is wrapped with the block ULID and aborts the LabelValues computation.","triggerScenarios":"The block's index header cache file is corrupt, was written by an incompatible thanos version, or was deleted/corrupted in the cache directory while the block is being queried.","commonSituations":"Upgrades where the on-disk index-header format changed; truncated downloads into the index-header cache; disk full or race during cache write; block deleted from bucket mid-query with stale cache.","solutions":["Clear the index-header cache directory for the affected block so it is re-fetched from object storage","Verify the object-store block index is intact (the underlying err names the real cause)","Restart the store gateway to rebuild the cache if entries are corrupt","Check disk space/permissions on the cache mount; upgrade thanos if the format changed across versions"],"exampleFix":"// before (cache entry corrupt, retry keeps failing)\n// rm -rf <cachedir>/index-header/<block-ULID> then restart store gateway\n// after: code-side handling\nres, err := indexr.block.indexHeaderReader.LabelValues(req.Label)\nif err != nil {\n    b.dropIndexHeaderCache() // force re-download\n    return errors.Wrapf(err, \"index header label values for block %s\", b.meta.ULID)\n}","handlingStrategy":"retry","validationCode":"if _, err := os.Stat(cacheIndexPath); err != nil || cacheCorrupt(cacheIndexPath) { invalidateCacheEntry() }","typeGuard":null,"tryCatchPattern":"res, err := indexr.block.indexHeaderReader.LabelValues(req.Label)\nif err != nil {\n    return errors.Wrapf(err, \"index header label values for block %s\", b.meta.ULID)\n}","preventionTips":["Monitor cache directory disk space and write failures","Clear stale index-header cache entries after thanos upgrades","Re-fetch from object storage on repeated cache read failures"],"tags":["object-storage","cache","index-header"],"backgroundTag":"file-read-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}