thanos-io/thanos · error

fetching targets from targets client

Error message

fetching targets from targets client %v

What it means

The targetsStream.receive fetches targets from the upstream Targets client and wraps any RPC failure with 'fetching targets from targets client %v'. Under PartialResponseStrategy_ABORT it returns the wrapped error; otherwise it forwards the wrapped error as a warning response to the downstream server and only fails if sending that warning also fails.

Solutions

1. Check connectivity/auth to the upstream targets endpoint referenced by stream.client
2. Inspect the wrapped cause to see the upstream gRPC status
3. If using ABORT strategy, add retry with backoff for transient upstream errors
4. If partial-response is acceptable, keep the current strategy and monitor the warning responses instead of failing

if err := stream.receive(ctx); err != nil { if strategy == ABORT { return err } /* warning already forwarded; degrade gracefully */ }

When it happens

Trigger: stream.client.Targets(ctx, request) returns an error during receive — upstream targets API unavailable, invalid request rejected by upstream, context cancellation, or timeout.

Common situations: Targets discovery backend (e.g. a store with targets API) down or unreachable, wrong address configured for the targets client, transient network errors in meshes where partial-response strategy tolerates failures.

Related errors

• Unknown
• proxy Exemplars
• proxy MetricMetadata
• fetching metric metadata from metadata client
• proxy Rules

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/de6806ce058bd54e. Report an issue: GitHub.

Appendix: source

Thrown at pkg/targets/proxy.go:93

		if err := srv.Send(targetspb.NewTargetsResponse(t)); err != nil {
			return status.Error(codes.Unknown, errors.Wrap(err, "send targets response").Error())
		}
	}

	return nil
}

type targetsStream struct {
	client  targetspb.TargetsClient
	request *targetspb.TargetsRequest
	channel chan<- *targetspb.TargetDiscovery
	server  targetspb.Targets_TargetsServer
}

func (stream *targetsStream) receive(ctx context.Context) error {
	targets, err := stream.client.Targets(ctx, stream.request)
	if err != nil {
		err = errors.Wrapf(err, "fetching targets from targets client %v", stream.client)

		if stream.request.PartialResponseStrategy == storepb.PartialResponseStrategy_ABORT {
			return err
		}

		if serr := stream.server.Send(targetspb.NewWarningTargetsResponse(err)); serr != nil {
			return serr
		}
		// Not an error if response strategy is warning.
		return nil
	}

	for {
		target, err := targets.Recv()
		if err == io.EOF {
			return nil
		}

View on GitHub (pinned to 35b8b99117)