{"record":{"id":"f612d40799512c85","repo":"thanos-io/thanos","slug":"copy-response","errorCode":null,"errorMessage":"copy response","messagePattern":"copy response","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/store/prometheus.go","lineNumber":401,"sourceCode":"\nfunc (s *BytesCounter) Read(p []byte) (n int, err error) {\n\tn, err = s.ReadCloser.Read(p)\n\ts.bytesCount += n\n\treturn n, err\n}\n\nfunc (s *BytesCounter) BytesCount() int {\n\treturn s.bytesCount\n}\n\nfunc (p *PrometheusStore) fetchSampledResponse(ctx context.Context, resp *http.Response) (_ *prompb.ReadResponse, err error) {\n\tdefer runutil.ExhaustCloseWithLogOnErr(p.logger, resp.Body, \"prom series request body\")\n\n\tb := p.getBuffer()\n\tbuf := bytes.NewBuffer(*b)\n\tdefer p.putBuffer(b)\n\tif _, err := io.Copy(buf, resp.Body); err != nil {\n\t\treturn nil, errors.Wrap(err, \"copy response\")\n\t}\n\n\tsb := p.getBuffer()\n\tvar decomp []byte\n\ttracing.DoInSpan(ctx, \"decompress_response\", func(ctx context.Context) {\n\t\tdecomp, err = snappy.Decode(*sb, buf.Bytes())\n\t})\n\tdefer p.putBuffer(sb)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"decompress response\")\n\t}\n\n\tvar data prompb.ReadResponse\n\ttracing.DoInSpan(ctx, \"unmarshal_response\", func(ctx context.Context) {\n\t\terr = proto.Unmarshal(decomp, &data)\n\t})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unmarshal response\")","sourceCodeStart":383,"sourceCodeEnd":419,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/store/prometheus.go#L383-L419","documentation":"fetchSampledResponse reads the entire sampled remote-read body with io.Copy into a pooled buffer; failure is wrapped as 'copy response'. This covers reading the HTTP response body from the Prometheus sampled (x-protobuf) remote read endpoint.","triggerScenarios":"Calling Series on a PrometheusStore using sampled remote read when the response body cannot be fully read: connection reset, unexpected EOF, timeouts mid-body, or server aborting the response.","commonSituations":"Large sampled responses cut off by proxy timeouts; network flakiness; Prometheus closing connections under load; keep-alive races.","solutions":["Check the wrapped cause for reset vs timeout and fix the underlying network/timeout setting.","Reduce the time range or step to shrink the sampled response size.","Raise proxy/LB response timeouts for long remote-read requests.","Prefer streamed chunked remote read (supported content type) for large series sets.","Retry the query; transient body-read failures often succeed on retry."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"data, err := s.fetchSampledResponse(ctx, resp, ...)\nif err != nil {\n    if strings.Contains(err.Error(), \"copy response\") {\n        // body read interrupted: retry; check errors.Unwrap for unexpected EOF/reset\n    }\n    return err\n}","preventionTips":["Bound query scope so sampled responses stay small.","Increase proxy/LB response timeout and disable premature connection recycling.","Prefer streamed chunked remote read for large result sets.","Monitor network errors between Thanos and Prometheus."],"tags":["prometheus","remote-read","http","network"],"backgroundTag":"response-body-read-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}