{"record":{"id":"0a899c0ab431c334","repo":"thanos-io/thanos","slug":"request-metric-against-s","errorCode":null,"errorMessage":"request metric against %s","messagePattern":"request metric against (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/promclient/promclient.go","lineNumber":715,"sourceCode":"// LowestTimestamp returns the lowest timestamp in the TSDB by parsing the /metrics endpoint\n// and extracting the prometheus_tsdb_lowest_timestamp_seconds metric from it.\nfunc (c *Client) LowestTimestamp(ctx context.Context, base *url.URL) (int64, error) {\n\tu := *base\n\tu.Path = path.Join(u.Path, \"/metrics\")\n\n\tlevel.Debug(c.logger).Log(\"msg\", \"lowest timestamp\", \"url\", u.String())\n\n\treq, err := http.NewRequest(http.MethodGet, u.String(), nil)\n\tif err != nil {\n\t\treturn 0, errors.Wrap(err, \"create request\")\n\t}\n\n\tspan, ctx := tracing.StartSpan(ctx, \"/lowest_timestamp HTTP[client]\")\n\tdefer span.Finish()\n\n\tresp, err := c.Do(req.WithContext(ctx))\n\tif err != nil {\n\t\treturn 0, errors.Wrapf(err, \"request metric against %s\", u.String())\n\t}\n\tdefer runutil.ExhaustCloseWithLogOnErr(c.logger, resp.Body, \"request body\")\n\n\tparser := expfmt.NewTextParser(model.UTF8Validation)\n\tfamilies, err := parser.TextToMetricFamilies(resp.Body)\n\tif err != nil {\n\t\treturn 0, errors.Wrapf(err, \"parsing metric families against %s\", u.String())\n\t}\n\tmf, ok := families[\"prometheus_tsdb_lowest_timestamp_seconds\"]\n\tif !ok {\n\t\treturn 0, errors.Wrapf(err, \"metric families did not contain 'prometheus_tsdb_lowest_timestamp_seconds'\")\n\t}\n\tval := 1000 * mf.GetMetric()[0].GetGauge().GetValue()\n\n\t// in the case that we dont have cut a block yet, TSDB lowest timestamp is math.MaxInt64\n\t// but its represented as float and truncated so we need to do this weird comparison.\n\t// Since we use this for fan-out pruning we use min timestamp here to include this prometheus.\n\tif val == float64(math.MaxInt64) {","sourceCodeStart":697,"sourceCodeEnd":733,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/promclient/promclient.go#L697-L733","documentation":"LowestTimestamp performs an HTTP GET against the Prometheus /metrics endpoint. If c.Do fails at the transport level (connection refused, DNS failure, TLS error, context canceled/timeout), the error is wrapped with \"request metric against <url>\".","triggerScenarios":"c.Do(req.WithContext(ctx)) errors: target Prometheus is down/unreachable, wrong host:port, TLS handshake failure, or the passed ctx is canceled/deadline-exceeded.","commonSituations":"Sidecar cannot reach the local Prometheus (wrong --prometheus.url), Prometheus restarted or crashed, network policy/firewall blocking the port, Kubernetes DNS issues, or query fan-out timing out.","solutions":["Verify the Prometheus URL and that the target is reachable (curl <url>/metrics) from the Thanos host/pod.","Check that Prometheus is running and listening on the configured port.","Inspect the wrapped cause: context.DeadlineExceeded means increase timeout or fix slow scrape; connection refused means wrong address.","Check network policies, service DNS names, and TLS settings between Thanos and Prometheus."],"exampleFix":"// before\nresp, err := c.Do(req.WithContext(ctx))\nif err != nil { return 0, errors.Wrapf(err, \"request metric against %s\", u.String()) }\n\n// after\n// Pre-check reachability and use a bounded context\nctx, cancel := context.WithTimeout(ctx, 10*time.Second)\ndefer cancel()\nif err := runutil.Retry(3, 2*time.Second, func() error {\n\t_, err := http.Get(u.String())\n\treturn err\n}); err != nil {\n\treturn err // surface persistent unreachability early\n}","handlingStrategy":"retry","validationCode":"// pre-check reachability\nif err := checkTCPReachable(host, port, 3*time.Second); err != nil {\n\treturn fmt.Errorf(\"prometheus %s unreachable: %w\", addr, err)\n}","typeGuard":null,"tryCatchPattern":"if err := runutil.Retry(3, 2*time.Second, func() error {\n\t_, err := client.LowestTimestamp(ctx, base)\n\treturn err\n}); err != nil {\n\tlevel.Warn(logger).Log(\"msg\", \"lowest timestamp unavailable\", \"err\", err)\n}","preventionTips":["Health-check the Prometheus endpoint before dependency on it","Set sensible request timeouts and retries for transient network faults","Monitor Prometheus liveness in the same network path as Thanos"],"tags":["network","http","prometheus","timeout"],"backgroundTag":"http-request-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}