{"record":{"id":"b6856ab2f3dca1d3","repo":"thanos-io/thanos","slug":"aborted","errorCode":"Aborted","errorMessage":"get exemplars for tenant %s","messagePattern":"get exemplars for tenant (.+?)","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/exemplars/multitsdb.go","lineNumber":38,"sourceCode":"\n// NewMultiTSDB creates new exemplars.MultiTSDB.\nfunc NewMultiTSDB(tsdbExemplarsServers func() map[string]*TSDB) *MultiTSDB {\n\treturn &MultiTSDB{\n\t\ttsdbExemplarsServers: tsdbExemplarsServers,\n\t}\n}\n\n// Exemplars returns all specified exemplars from a MultiTSDB instance.\nfunc (m *MultiTSDB) Exemplars(r *exemplarspb.ExemplarsRequest, s exemplarspb.Exemplars_ExemplarsServer) error {\n\texpr, err := extpromql.ParseExpr(r.Query)\n\tif err != nil {\n\t\treturn status.Error(codes.Internal, err.Error())\n\t}\n\tmatchers := parser.ExtractSelectors(expr)\n\n\tfor tenant, es := range m.tsdbExemplarsServers() {\n\t\tif err := es.Exemplars(matchers, r.Start, r.End, s); err != nil {\n\t\t\treturn status.Error(codes.Aborted, errors.Wrapf(err, \"get exemplars for tenant %s\", tenant).Error())\n\t\t}\n\t}\n\treturn nil\n}\n","sourceCodeStart":20,"sourceCodeEnd":43,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/exemplars/multitsdb.go#L20-L43","documentation":"In the multi-TSDB exemplars path, Exemplars iterates every tenant's TSDB exemplars server and wraps any per-tenant failure with errors.Wrapf(err, \"get exemplars for tenant %s\", tenant), returned as a gRPC Aborted status. It tells you which tenant's store failed while serving exemplars.","triggerScenarios":"A multi-tenant ThanOS ruler/query setup where es.Exemplars(matchers, r.Start, r.End, s) fails for one tenant — e.g. corrupt head block, exemplars API disabled, or time-range beyond retention.","commonSituations":"Multi-tenant deployments where one tenant's TSDB has an issue while others serve fine; queries spanning tenants hit the first failing tenant and abort the whole request.","solutions":["Identify the tenant named in the error message and inspect that tenant's TSDB logs for the wrapped cause.","Verify exemplars storage is enabled and healthy for that tenant (max_exemplars > 0).","Check the requested time range is within the tenant's retention/block layout.","Repair or reload the affected tenant's TSDB blocks if corruption is reported."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"_, err := api.Exemplars(ctx, req)\nif st, ok := status.FromError(err); ok && st.Code() == codes.Aborted {\n\t// message names the failing tenant; alert on that tenant's TSDB\n\tlog.Errorf(\"tenant exemplars failed: %v\", st.Message())\n}","preventionTips":["Set --storage.tsdb.max-exemplars > 0 on every tenant's store.","Monitor per-tenant TSDB health (head block, compactions).","Keep query time ranges within tenant retention."],"tags":["go","grpc","exemplars","multitenancy","tsdb"],"backgroundTag":"database-query-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}