thanos-io/thanos · error

sending metadata error to server

Error message

sending metadata error to server %v

What it means

When an upstream stream error must be relayed to the caller as a warning response, a failure of that relay Send is wrapped with 'sending metadata error to server %v'. This is a secondary failure: reporting the original error also failed.

Solutions

1. Treat as a client-disconnect if the wrapped status is Canceled; log and stop
2. Retry only if the client is still connected
3. Monitor for repeated occurrences — indicates upstream errors plus flaky clients
4. Check the wrapped original upstream error for the root cause

if err := relay(err); err != nil {
    if status.Code(err) == codes.Canceled {
        return nil // caller disconnected; nothing to do
    }
    return err
}

When it happens

Trigger: stream.server.Send(NewWarningMetadataResponse(err)) fails while handling a previous upstream error under the WARNING partial-response strategy.

Common situations: Caller disconnected right as the proxy tried to report the upstream failure; client stream already closed/canceled.

Related errors

• Unknown
• Unknown
• %s
• empty exemplars data
• res.GetWarning()

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/8e0a33004de43373. Report an issue: GitHub.

Appendix: source

Thrown at pkg/metadata/proxy.go:135

		// Not an error if response strategy is warning.
		return nil
	}

	for {
		resp, err := metadataCli.Recv()
		if err == io.EOF {
			return nil
		}

		if err != nil {
			err = errors.Wrapf(err, "receiving metric metadata from metadata client %v", stream.client)

			if stream.request.PartialResponseStrategy == storepb.PartialResponseStrategy_ABORT {
				return err
			}

			if err := stream.server.Send(metadatapb.NewWarningMetadataResponse(err)); err != nil {
				return errors.Wrapf(err, "sending metadata error to server %v", stream.server)
			}

			// Not an error if response strategy is warning.
			return nil
		}

		if w := resp.GetWarning(); w != "" {
			if err := stream.server.Send(metadatapb.NewWarningMetadataResponse(errors.New(w))); err != nil {
				return errors.Wrapf(err, "sending metadata warning to server %v", stream.server)
			}
			continue
		}

		select {
		case stream.channel <- resp.GetMetadata():
		case <-ctx.Done():
			return ctx.Err()
		}

View on GitHub (pinned to 35b8b99117)