thanos-io/thanos · error

sending exemplars warning to server

Error message

sending exemplars warning to server %v

What it means

The exemplars proxy received a warning from a store and tried to forward it as a warning response to the client, but stream.server.Send failed, so the downstream gRPC connection is unusable and the partial-response warning cannot be delivered.

Solutions

1. Treat the stream as dead and return; the client is gone or the transport broke.
2. Log with the stream.server identifier for diagnosis.
3. No retry: per-request streaming cannot be resumed.

When it happens

Trigger: Thrown at pkg/exemplars/proxy.go:212 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/d003fcb94cd5cd3a. Report an issue: GitHub.

Appendix: source

Thrown at pkg/exemplars/proxy.go:212

		}

		if err != nil {
			err = errors.Wrapf(err, "receiving exemplars from exemplars client %v", stream.client)

			if stream.request.PartialResponseStrategy == storepb.PartialResponseStrategy_ABORT {
				return err
			}

			if err := stream.server.Send(exemplarspb.NewWarningExemplarsResponse(err)); err != nil {
				return errors.Wrapf(err, "sending exemplars error to server %v", stream.server)
			}
			// Not an error if response strategy is warning.
			return nil
		}

		if w := exemplar.GetWarning(); w != "" {
			if err := stream.server.Send(exemplarspb.NewWarningExemplarsResponse(errors.New(w))); err != nil {
				return errors.Wrapf(err, "sending exemplars warning to server %v", stream.server)
			}
			continue
		}

		select {
		case stream.channel <- exemplar.GetData():
		case <-ctx.Done():
			return ctx.Err()
		}
	}
}

// matchesExternalLabels returns false if given matchers are not matching external labels.
// If true, matchesExternalLabels also returns Prometheus matchers without those matching external labels.
func matchesExternalLabels(ms []*labels.Matcher, externalLabels labels.Labels) (bool, []*labels.Matcher) {
	if externalLabels.IsEmpty() {
		return true, ms
	}

View on GitHub (pinned to 35b8b99117)