thanos-io/thanos · error

roundtripping to downstream URL

Error message

roundtripping to downstream URL %s

What it means

The periodic downstream readiness probe's HTTP RoundTrip against <downstreamURL>/-/ready failed, meaning the query-frontend cannot reach the downstream query service (network error, DNS failure, TLS problem, or timeout).

Solutions

1. Verify the downstream URL flag points to a reachable query instance
2. Check network/DNS/firewall between frontend and downstream
3. Confirm downstream is running and /-/ready responds
4. Check TLS configuration on both sides

When it happens

Trigger: Thrown at cmd/thanos/query_frontend.go:421 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/cc7926bd9f6e4455. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/query_frontend.go:421

	{
		ctx, cancel := context.WithCancel(context.Background())

		g.Add(func() error {
			var firstRun = true

			doCheckDownstream := func() (rerr error) {
				timeoutCtx, cancel := context.WithTimeout(ctx, 10*time.Second)
				defer cancel()

				readinessUrl := cfg.DownstreamURL + "/-/ready"
				req, err := http.NewRequestWithContext(timeoutCtx, http.MethodGet, readinessUrl, nil)
				if err != nil {
					return errors.Wrap(err, "creating request to downstream URL")
				}

				resp, err := downstreamRT.RoundTrip(req)
				if err != nil {
					return errors.Wrapf(err, "roundtripping to downstream URL %s", readinessUrl)
				}
				defer runutil.CloseWithErrCapture(&rerr, resp.Body, "downstream health check response body")

				if resp.StatusCode/100 == 4 || resp.StatusCode/100 == 5 {
					return errors.Errorf("downstream URL %s returned an error: %d", readinessUrl, resp.StatusCode)
				}

				return nil
			}
			for {
				if !firstRun {
					select {
					case <-ctx.Done():
						return nil
					case <-time.After(10 * time.Second):
					}
				}
				firstRun = false

View on GitHub (pinned to 35b8b99117)