thanos-io/thanos · error

receiving metric metadata from metadata client

Error message

receiving metric metadata from metadata client %v

What it means

In the metadata proxy receive loop, metadataCli.Recv returned a non-EOF error while streaming metric metadata from a store API, so the metadata result is incomplete. Under ABORT strategy the error aborts the request; under warning strategy it is forwarded as a warning response and this stream stops for the client.

Solutions

1. Retry the metadata request; transient stream breakage often resolves
2. Check upstream service logs for the reason the stream terminated
3. Verify load balancer idle timeouts are not killing long gRPC streams
4. Use the appropriate PartialResponseStrategy: ABORT propagates the error, WARNING converts it to a warning for the caller

err := consumeMetadataStream(ctx)
if err != nil {
    if isTransient(status.Code(err)) { // Unavailable, Canceled on upstream restart
        retryWithBackoff(consumeMetadataStream)
    }
}

When it happens

Trigger: Upstream closes the stream unexpectedly, a gRPC error status arrives mid-stream, or the context is canceled while receiving.

Common situations: Vizier restart during a metadata query; network interruption on a long-running metadata scan; keepalive failures terminating the stream.

Related errors

• sending rules warning to server
• Unknown
• receiving tsdb statistics from status client
• iterate series for block
• send series response

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/0743bebbb54957fc. Report an issue: GitHub.

Appendix: source

Thrown at pkg/metadata/proxy.go:128

		if stream.request.PartialResponseStrategy == storepb.PartialResponseStrategy_ABORT {
			return err
		}

		if serr := stream.server.Send(metadatapb.NewWarningMetadataResponse(err)); serr != nil {
			return serr
		}
		// Not an error if response strategy is warning.
		return nil
	}

	for {
		resp, err := metadataCli.Recv()
		if err == io.EOF {
			return nil
		}

		if err != nil {
			err = errors.Wrapf(err, "receiving metric metadata from metadata client %v", stream.client)

			if stream.request.PartialResponseStrategy == storepb.PartialResponseStrategy_ABORT {
				return err
			}

			if err := stream.server.Send(metadatapb.NewWarningMetadataResponse(err)); err != nil {
				return errors.Wrapf(err, "sending metadata error to server %v", stream.server)
			}

			// Not an error if response strategy is warning.
			return nil
		}

		if w := resp.GetWarning(); w != "" {
			if err := stream.server.Send(metadatapb.NewWarningMetadataResponse(errors.New(w))); err != nil {
				return errors.Wrapf(err, "sending metadata warning to server %v", stream.server)
			}
			continue

View on GitHub (pinned to 35b8b99117)