{"record":{"id":"f6932e2728ec5599","repo":"thanos-io/thanos","slug":"query-prometheus","errorCode":null,"errorMessage":"query Prometheus","messagePattern":"query Prometheus","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/store/prometheus.go","lineNumber":206,"sourceCode":"\t\tcase labels.MatchNotEqual:\n\t\t\tpm.Type = prompb.LabelMatcher_NEQ\n\t\tcase labels.MatchRegexp:\n\t\t\tpm.Type = prompb.LabelMatcher_RE\n\t\tcase labels.MatchNotRegexp:\n\t\t\tpm.Type = prompb.LabelMatcher_NRE\n\t\tdefault:\n\t\t\treturn errors.New(\"unrecognized matcher type\")\n\t\t}\n\t\tq.Matchers = append(q.Matchers, pm)\n\t}\n\n\tqueryPrometheusSpan, ctx := tracing.StartSpan(s.Context(), \"query_prometheus\")\n\tqueryPrometheusSpan.SetTag(\"query.request\", q.String())\n\n\thttpResp, err := p.startPromRemoteRead(ctx, q)\n\tif err != nil {\n\t\tqueryPrometheusSpan.Finish()\n\t\treturn errors.Wrap(err, \"query Prometheus\")\n\t}\n\n\t// Negotiate content. We requested streamed chunked response type, but still we need to support old versions of\n\t// remote read.\n\tcontentType := httpResp.Header.Get(\"Content-Type\")\n\tif strings.HasPrefix(contentType, \"application/x-protobuf\") {\n\t\treturn p.handleSampledPrometheusResponse(s, httpResp, queryPrometheusSpan, extLset, enableChunkHashCalculation, extLsetToRemove)\n\t}\n\n\tif !strings.HasPrefix(contentType, \"application/x-streamed-protobuf; proto=prometheus.ChunkedReadResponse\") {\n\t\treturn errors.Errorf(\"not supported remote read content type: %s\", contentType)\n\t}\n\treturn p.handleStreamedPrometheusResponse(s, shardMatcher, httpResp, queryPrometheusSpan, extLset, enableChunkHashCalculation, extLsetToRemove)\n}\n\nfunc (p *PrometheusStore) handleSampledPrometheusResponse(\n\ts flushableServer,\n\thttpResp *http.Response,","sourceCodeStart":188,"sourceCodeEnd":224,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/store/prometheus.go#L188-L224","documentation":"Series issues the remote-read HTTP request via p.startPromRemoteRead; if that returns an error, it is wrapped as 'query Prometheus'. This covers request construction, URL building, connection, and non-handled transport failures against the Prometheus remote read endpoint.","triggerScenarios":"Calling Series on a PrometheusStore whose endpoint is unreachable, has a bad URL/DNS name, is not serving /api/v1/read, TLS verification fails, or the request times out.","commonSituations":"Wrong Prometheus address in the store-gateway/proxy config; Prometheus not exposing remote read (no --enable-feature=remote-write-receiver variants or old version); network policies/firewalls; TLS/mTLS misconfiguration.","solutions":["Verify the Prometheus remote-read endpoint URL and that /api/v1/read responds (curl it from the store host).","Check network connectivity, DNS, firewalls, and service discovery between Thanos and Prometheus.","Confirm TLS settings (CA bundle, client certs) match the Prometheus server configuration.","Check the wrapped cause (%w) for timeout vs connection-refused details and adjust timeouts."],"exampleFix":"// before\np, _ := NewPrometheusStore(..., \"http://prometheus:9090/reading\", ...)\n// after\np, _ := NewPrometheusStore(..., \"http://prometheus:9090/api/v1/read\", ...)","handlingStrategy":"retry","validationCode":"u, err := url.Parse(remoteReadURL)\nif err != nil || u.Scheme == \"\" || u.Host == \"\" {\n    return fmt.Errorf(\"invalid remote read URL %q\", remoteReadURL)\n}\nresp, err := http.Get(u.String() /* or HEAD on /api/v1/read */) // verify reachability first","typeGuard":null,"tryCatchPattern":"if err := s.Series(ctx, req); err != nil {\n    if strings.Contains(err.Error(), \"query Prometheus\") {\n        // retry with backoff for transient network errors; check errors.Unwrap for connection-refused vs timeout\n    }\n    return err\n}","preventionTips":["Health-check the Prometheus remote-read endpoint before querying.","Use correct /api/v1/read path and scheme in configuration.","Set explicit, generous timeouts for remote read requests.","Validate TLS/CA configuration in staging first."],"tags":["prometheus","remote-read","http","network"],"backgroundTag":"http-request-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}