thanos-io/thanos · error · ApiError

retrieving targets

Error message

retrieving targets

What it means

This error wraps a failure from client.Targets() — the Query API's call to the Query Frontend/upstream store-gateway Targets RPC used by the /api/v1/targets endpoint. The underlying error is whatever the gRPC client or downstream Thanos Query node reported, wrapped as ErrorInternal (HTTP 500). The handler itself is fine; the failure is in retrieving target data from the cluster.

Solutions

1. Inspect the wrapped err message for the root cause (connection refused, DeadlineExceeded, canceled, etc.).
2. Verify the Thanos Query nodes behind the frontend are healthy and reachable (grpc port, --query addresses).
3. Retry the request if the error was a timeout or transient network issue.
4. Check frontend-to-query TLS/cert configuration if the error indicates TLS handshake failure.

await Promise.race([fetch(url, {signal: AbortSignal.timeout(30000)}), ...]) // enforce client timeout below server deadline

try { const r = await fetch('/api/v1/targets'); const b = await r.json(); if (b.status === 'error') handleApiError(b.error); } catch (e) { if (isTimeout(e)) retryWithBackoff(); else throw e; }

When it happens

Trigger: The proxied Targets(ctx, req) gRPC call returns a non-nil err — e.g. the query node is unreachable, the context deadline/r.Context() was canceled, or the store-gateway Targets API returns an RPC error.

Common situations: Query node down or DNS resolvable but gRPC port unreachable, client request canceled/timed out mid-call, TLS or auth misconfiguration between query-frontend and query, or the downstream query node returning 'no store matched' style RPC failures.

Related errors

• error retrieving rules
• retrieving exemplars
• no query API server reachable
• failed to get tsdb status from prometheus
• proxy MetricMetadata

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/c887ced3d684c6d4. Report an issue: GitHub.

Appendix: source

Thrown at pkg/api/query/v1.go:1400

	return func(r *http.Request) (any, []error, *api.ApiError, func()) {
		stateParam := r.URL.Query().Get("state")
		state, ok := targetspb.TargetsRequest_State_value[strings.ToUpper(stateParam)]
		if !ok {
			if stateParam != "" {
				return nil, nil, &api.ApiError{Typ: api.ErrorBadData, Err: errors.Errorf("invalid targets parameter state='%v'", stateParam)}, func() {}
			}
			state = int32(targetspb.TargetsRequest_ANY)
		}

		req := &targetspb.TargetsRequest{
			State:                   targetspb.TargetsRequest_State(state),
			PartialResponseStrategy: ps,
		}

		t, warnings, err := client.Targets(r.Context(), req)
		if err != nil {
			return nil, nil, &api.ApiError{Typ: api.ErrorInternal, Err: errors.Wrap(err, "retrieving targets")}, func() {}
		}

		return t, warnings.AsErrors(), nil, func() {}
	}
}

// NewAlertsHandler created handler compatible with HTTP /api/v1/alerts https://prometheus.io/docs/prometheus/latest/querying/api/#alerts
// which uses gRPC Unary Rules API (Rules API works for both /alerts and /rules).
func NewAlertsHandler(client rules.UnaryClient, enablePartialResponse bool) func(*http.Request) (any, []error, *api.ApiError, func()) {
	ps := storepb.PartialResponseStrategy_ABORT
	if enablePartialResponse {
		ps = storepb.PartialResponseStrategy_WARN
	}

	return func(r *http.Request) (any, []error, *api.ApiError, func()) {
		span, ctx := tracing.StartSpan(r.Context(), "receive_http_request")
		defer span.Finish()

View on GitHub (pinned to 35b8b99117)