{"record":{"id":"0743bebbb54957fc","repo":"thanos-io/thanos","slug":"receiving-metric-metadata-from-metadata-client-v","errorCode":null,"errorMessage":"receiving metric metadata from metadata client %v","messagePattern":"receiving metric metadata from metadata client (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/metadata/proxy.go","lineNumber":128,"sourceCode":"\t\tif stream.request.PartialResponseStrategy == storepb.PartialResponseStrategy_ABORT {\n\t\t\treturn err\n\t\t}\n\n\t\tif serr := stream.server.Send(metadatapb.NewWarningMetadataResponse(err)); serr != nil {\n\t\t\treturn serr\n\t\t}\n\t\t// Not an error if response strategy is warning.\n\t\treturn nil\n\t}\n\n\tfor {\n\t\tresp, err := metadataCli.Recv()\n\t\tif err == io.EOF {\n\t\t\treturn nil\n\t\t}\n\n\t\tif err != nil {\n\t\t\terr = errors.Wrapf(err, \"receiving metric metadata from metadata client %v\", stream.client)\n\n\t\t\tif stream.request.PartialResponseStrategy == storepb.PartialResponseStrategy_ABORT {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tif err := stream.server.Send(metadatapb.NewWarningMetadataResponse(err)); err != nil {\n\t\t\t\treturn errors.Wrapf(err, \"sending metadata error to server %v\", stream.server)\n\t\t\t}\n\n\t\t\t// Not an error if response strategy is warning.\n\t\t\treturn nil\n\t\t}\n\n\t\tif w := resp.GetWarning(); w != \"\" {\n\t\t\tif err := stream.server.Send(metadatapb.NewWarningMetadataResponse(errors.New(w))); err != nil {\n\t\t\t\treturn errors.Wrapf(err, \"sending metadata warning to server %v\", stream.server)\n\t\t\t}\n\t\t\tcontinue","sourceCodeStart":110,"sourceCodeEnd":146,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/metadata/proxy.go#L110-L146","documentation":"In the metadata proxy receive loop, metadataCli.Recv returned a non-EOF error while streaming metric metadata from a store API, so the metadata result is incomplete. Under ABORT strategy the error aborts the request; under warning strategy it is forwarded as a warning response and this stream stops for the client.","triggerScenarios":"Upstream closes the stream unexpectedly, a gRPC error status arrives mid-stream, or the context is canceled while receiving.","commonSituations":"Vizier restart during a metadata query; network interruption on a long-running metadata scan; keepalive failures terminating the stream.","solutions":["Retry the metadata request; transient stream breakage often resolves","Check upstream service logs for the reason the stream terminated","Verify load balancer idle timeouts are not killing long gRPC streams","Use the appropriate PartialResponseStrategy: ABORT propagates the error, WARNING converts it to a warning for the caller"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"err := consumeMetadataStream(ctx)\nif err != nil {\n    if isTransient(status.Code(err)) { // Unavailable, Canceled on upstream restart\n        retryWithBackoff(consumeMetadataStream)\n    }\n}","preventionTips":["Configure gRPC keepalive and LB idle timeouts above stream duration","Retry metadata queries that break mid-stream","Watch upstream logs during Vizier restarts"],"tags":["grpc","streaming","network"],"backgroundTag":"upstream-api-error","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}