{"record":{"id":"63797e8a31529453","repo":"thanos-io/thanos","slug":"lookup-labels-symbols","errorCode":null,"errorMessage":"Lookup labels symbols","messagePattern":"Lookup labels symbols","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/store/bucket.go","lineNumber":1327,"sourceCode":"\tseriesMatched := 0\n\tb.entries = b.entries[:0]\nOUTER:\n\tfor i := range postingsBatch {\n\t\tif err := b.ctx.Err(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\thasMatchedChunks, err := b.indexr.LoadSeriesForTime(postingsBatch[i], &b.symbolizedLset, &b.chkMetas, b.skipChunks, b.mint, b.maxt)\n\t\tif err != nil {\n\t\t\treturn errors.Wrap(err, \"read series\")\n\t\t}\n\t\t// Skip getting series symbols if we know there is no matched chunks\n\t\t// and lazy expanded posting not enabled.\n\t\tif !lazyExpandedPosting && !hasMatchedChunks {\n\t\t\tcontinue\n\t\t}\n\n\t\tif err := b.indexr.LookupLabelsSymbols(b.ctx, b.symbolizedLset, b.b); err != nil {\n\t\t\treturn errors.Wrap(err, \"Lookup labels symbols\")\n\t\t}\n\t\tb.lset = b.b.Labels()\n\n\t\tfor _, matcher := range b.lazyPostings.matchers {\n\t\t\tval := b.lset.Get(matcher.Name)\n\t\t\tif !matcher.Matches(val) {\n\t\t\t\t// Series not matched means series we overfetched due to lazy posting expansion.\n\t\t\t\tseriesBytes := b.indexr.loadedSeries[postingsBatch[i]]\n\t\t\t\tb.lazyExpandedPostingSeriesOverfetchedSizeBytes.Add(float64(len(seriesBytes)))\n\t\t\t\tcontinue OUTER\n\t\t\t}\n\t\t}\n\t\tif lazyExpandedPosting {\n\t\t\tb.expandedPostings = append(b.expandedPostings, postingsBatch[i])\n\t\t}\n\t\t// If lazy expanded postings enabled, due to expanded postings cache, we need to\n\t\t// make sure we check lazy posting matchers and update expanded postings before\n\t\t// going to next series.","sourceCodeStart":1309,"sourceCodeEnd":1345,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/store/bucket.go#L1309-L1345","documentation":"Wraps IndexReader.LookupLabelsSymbols failure in blockSeriesClient.series. After loading a series' label set in symbolized form, this resolves symbol references to actual label strings. Failure means symbol table lookups against the block index failed (fetch/decode error or out-of-range symbol reference).","triggerScenarios":"Series with matched chunks (or lazy expanded posting) call indexr.LookupLabelsSymbols(ctx, symbolizedLset, b) and the symbol table lookup or fetch fails; also occurs when a series references symbols outside the symbol table (index corruption).","commonSituations":"Corrupted or truncated symbol tables in uploaded blocks, object-store fetch failures under load, block deleted from the bucket while a query was streaming it.","solutions":["Verify the block's index with 'thanos tools bucket verify'; remove/restore corrupted blocks.","Check the wrapped root error for object-store connectivity/throttling and retry the query.","Ensure blocks are not deleted while store gateways still sync/serve them (retain long enough, coordinate compactor deletion).","Rebuild the index-header if symbol table metadata is stale."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := query(ctx); err != nil && strings.Contains(err.Error(), \"Lookup labels symbols\") {\n\tif isNotFound(err) {\n\t\t// block likely deleted mid-query: refresh store list and retry once\n\t\tif retryErr := query(ctx); retryErr != nil {\n\t\t\treturn retryErr\n\t\t}\n\t\treturn nil\n\t}\n\treturn err\n}","preventionTips":["Coordinate compactor deletions with store gateways (sufficient block retention/sync delay).","Verify blocks to catch corrupted symbol tables before serving queries.","Monitor object-store fetch errors from store gateways.","Rebuild index-headers when symbol-table metadata is suspect."],"tags":["index","symbols","thanos-store"],"backgroundTag":"database-query-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}