thanos-io/thanos · error

perform request against

Error message

perform %s request against %s

What it means

req2xx failed to execute the HTTP request against the Prometheus API: c.Do(req) returned a transport-level error. The message includes the method and target URL. This happens before any response status is available — DNS failure, connection refused, TLS error, or context cancellation.

Solutions

1. Check Prometheus is up and listening on the configured URL (curl the URL from the same host)
2. Inspect the wrapped error: connection refused vs DNS vs timeout indicate different fixes
3. Increase client timeout / verify ctx cancellation isn't premature
4. Fix DNS, network policy, or TLS trust configuration as indicated by the cause

Example fix

// before
resp, err := client.QueryInstant(ctx, apiCtx, nil)
// after
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
resp, err := client.QueryInstant(ctx, apiCtx, nil)
if err != nil && status.Code(err) == codes.DeadlineExceeded { /* retry with backoff */ }

err := runutil.Retry(2*time.Second, ctx.Done(), func() error {
    _, _, err := client.BuildVersion(ctx, promURL)
    if err != nil && !errors.Is(err, context.Canceled) {
        return err // transport failure: retry
    }
    return nil
})

When it happens

Trigger: Any client call (QueryInstant, QueryRange, ExternalLabels, BuildVersion, etc.) where the Prometheus host is unreachable, the port is wrong, TLS fails, or ctx is cancelled mid-request.

Common situations: Prometheus down or restarted; sidecar pointed at the wrong address/port; network policy or firewall blocking egress; request context deadline exceeded on slow queries.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• failed to validate prometheus flags
• request config against
• read query instant response
• read query range response
• request metric against

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/e7fac1467bd128de. Report an issue: GitHub.

Appendix: source

Thrown at pkg/promclient/promclient.go:135

	req, err := http.NewRequest(method, u.String(), b)
	if err != nil {
		return nil, 0, errors.Wrapf(err, "create %s request", method)
	}
	if headers != nil {
		req.Header = headers
	}

	if c.userAgent != "" {
		req.Header.Set("User-Agent", c.userAgent)
	}
	if method == http.MethodPost {
		req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
	}

	resp, err := c.Do(req.WithContext(ctx))
	if err != nil {
		return nil, 0, errors.Wrapf(err, "perform %s request against %s", method, u.String())
	}
	defer runutil.ExhaustCloseWithErrCapture(&err, resp.Body, "%s: close body", req.URL.String())

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, resp.StatusCode, errors.Wrap(err, "read body")
	}
	if resp.StatusCode/100 != 2 {
		return nil, resp.StatusCode, errors.Errorf("expected 2xx response, got %d. Body: %v", resp.StatusCode, string(body))
	}
	return body, resp.StatusCode, nil
}

// IsWALDirAccessible returns no error if WAL dir can be found. This helps to tell
// if we have access to Prometheus TSDB directory.
func IsWALDirAccessible(dir string) error {
	const errMsg = "WAL dir is not accessible. Is this dir a TSDB directory? If yes it is shared with TSDB?"

View on GitHub (pinned to 35b8b99117)