thanos-io/thanos · error

sending tsdb statistics warning to server

Error message

sending tsdb statistics warning to server %v

What it means

When the upstream statistics stream sends a warning message (resp.GetWarning() non-empty), the proxy relays it to the caller. If `stream.server.Send(...)` for that warning fails, the send error is wrapped with this message. The warning content itself is upstream store chatter; the error is purely a failed delivery of it.

Solutions

1. Check why the client connection to the proxy dropped (timeouts, cancellation).
2. Reduce warning volume by fixing the offending store that emits them.
3. Verify LB/proxy idle timeouts allow the full statistics stream.
4. Retry the request once the client connection is healthy.

Example fix

// before
client with 5s deadline on a stream taking 15s
// after
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)

if err := call(ctx); err != nil {
    if strings.Contains(err.Error(), "sending tsdb statistics warning to server") {
        logger.Debug("failed to relay warning to disconnected client", "err", err)
        return nil
    }
    return err
}

When it happens

Trigger: receive() loop hits a warning response from the store and attempts `stream.server.Send(NewWarningTSDBStatisticsResponse(...))` while the downstream client stream is dead/cancelled.

Common situations: Client timeout or disconnect while stores emit warnings; many warnings from a misbehaving store; load balancer terminating long-lived streams.

Related errors

• receiving tsdb statistics from status client
• sending tsdb statistics error to server
• Unknown
• %s
• empty exemplars data

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/1da98424d659c533. Report an issue: GitHub.

Appendix: source

Thrown at pkg/status/proxy.go:139

		if err != nil {
			err = errors.Wrapf(err, "receiving tsdb statistics from status client %v", stream.client)

			if stream.request.PartialResponseStrategy == storepb.PartialResponseStrategy_ABORT {
				return err
			}

			if err := stream.server.Send(statuspb.NewWarningTSDBStatisticsResponse(err)); err != nil {
				return errors.Wrapf(err, "sending tsdb statistics error to server %v", stream.server)
			}

			// Not an error if response strategy is warning.
			return nil
		}

		if w := resp.GetWarning(); w != "" {
			if err := stream.server.Send(statuspb.NewWarningTSDBStatisticsResponse(errors.New(w))); err != nil {
				return errors.Wrapf(err, "sending tsdb statistics warning to server %v", stream.server)
			}
			continue
		}

		select {
		case stream.channel <- resp.GetStatistics():
		case <-ctx.Done():
			return ctx.Err()
		}
	}
}

View on GitHub (pinned to 35b8b99117)