{"record":{"id":"c9cbb53e54bc4d85","repo":"thanos-io/thanos","slug":"proxy-tsdbstatistics","errorCode":null,"errorMessage":"proxy TSDBStatistics","messagePattern":"proxy TSDBStatistics","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/status/status.go","lineNumber":48,"sourceCode":"\tproxy statuspb.StatusServer\n}\n\nfunc NewGRPCClient(ss statuspb.StatusServer) *GRPCClient {\n\treturn &GRPCClient{\n\t\tproxy: ss,\n\t}\n}\n\nfunc (gc *GRPCClient) TSDBStatistics(ctx context.Context, req *statuspb.TSDBStatisticsRequest) (map[string]*statuspb.TSDBStatisticsEntry, annotations.Annotations, error) {\n\tspan, ctx := tracing.StartSpan(ctx, \"tsdb_statistics_grpc_request\")\n\tdefer span.Finish()\n\n\tsrv := &tsdbStatisticsServer{\n\t\tctx:            ctx,\n\t\ttsdbStatistics: map[string][]*statuspb.TSDBStatisticsEntry{},\n\t}\n\tif err := gc.proxy.TSDBStatistics(req, srv); err != nil {\n\t\treturn nil, nil, errors.Wrap(err, \"proxy TSDBStatistics\")\n\t}\n\n\t// Collect and aggregate the results.\n\ttsdbStatistics := make(map[string]*statuspb.TSDBStatisticsEntry)\n\tfor tenant, stats := range srv.tsdbStatistics {\n\t\ttenantStats, found := tsdbStatistics[tenant]\n\t\tif !found {\n\t\t\ttenantStats = &statuspb.TSDBStatisticsEntry{}\n\t\t\ttsdbStatistics[tenant] = tenantStats\n\t\t}\n\n\t\tfor _, stat := range stats {\n\t\t\ttenantStats.Merge(stat)\n\t\t}\n\t}\n\n\treturn tsdbStatistics, srv.warnings, nil\n}","sourceCodeStart":30,"sourceCodeEnd":66,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/status/status.go#L30-L66","documentation":"The querier/gateway TSDBStatistics fan-out calls the wrapped gossip/proxy (`gc.proxy.TSDBStatistics`) which aggregates responses from all peers; any error from the proxy is wrapped as 'proxy TSDBStatistics'. It means the distributed statistics aggregation itself failed (e.g. one member's stream errored under ABORT strategy).","triggerScenarios":"Calling (Store|Bucket) TSDBStatistics API which invokes `gc.proxy.TSDBStatistics(req, srv)` and receives an error — typically propagated from a member/store stream failure like errors 922-927.","commonSituations":"One store node down while using ABORT partial response strategy; peer-to-peer proxy connection failure in store-gateway clusters; request timeouts during large aggregations.","solutions":["Inspect the wrapped cause — usually a failed store/member stream (see underlying error).","Use PartialResponseStrategy WARNINGS so a single unhealthy peer doesn't fail the aggregation.","Restore the failed store node or remove it from the peer set.","Increase request timeouts for clusters with many stores."],"exampleFix":"// before\nreq := &statuspb.TSDBStatisticsRequest{PartialResponseStrategy: storepb.PartialResponseStrategy_ABORT}\n// after\nreq := &statuspb.TSDBStatisticsRequest{PartialResponseStrategy: storepb.PartialResponseStrategy_WARNINGS}","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"res, warnings, err := api.TSDBStatistics(ctx, req)\nif err != nil {\n    if strings.Contains(err.Error(), \"proxy TSDBStatistics\") {\n        // fall back to per-store direct queries skipping bad peers\n        return collectDirectFromHealthyStores(ctx)\n    }\n    return err\n}","preventionTips":["Prefer WARNINGS strategy to tolerate a bad peer.","Keep all cluster members healthy and current.","Alert on peer connectivity in store-gateway clusters.","Size timeouts for the largest expected aggregation."],"tags":["grpc","proxy","aggregation","fan-out"],"backgroundTag":"upstream-api-error","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}