thanos-io/thanos · error
empty exemplars data
Error message
empty exemplars data
What it means
exemplarsServer.Send returns this error when an ExemplarsResponse arrives with no warning but a nil Data payload. Such a response carries nothing usable, so the stream treats it as a protocol violation rather than silently appending nil data.
Solutions
- Fix the sender to always wrap data with exemplarspb.NewExemplarsResponse(data) before Send.
- If a store genuinely has no data, it should send no response rather than an empty one.
- Check for version mismatches between client and store (older ThanOS stores with different response shapes).
- In tests, construct responses via the NewExemplarsResponse helper instead of raw struct literals.
Example fix
// before (sender side)
stream.Send(&exemplarspb.ExemplarsResponse{})
// after
stream.Send(exemplarspb.NewExemplarsResponse(data)) Defensive patterns
Strategy: validation
Validate before calling
func validResponse(res *exemplarspb.ExemplarsResponse) bool {
return res.GetWarning() != "" || res.GetData() != nil
} Type guard
func hasExemplarData(res *exemplarspb.ExemplarsResponse) bool {
return res != nil && res.GetData() != nil
} Prevention
- Only send responses via exemplarspb.NewExemplarsResponse(data).
- Send nothing when there is no data rather than an empty response.
- Cover senders with tests asserting non-nil Data.
When it happens
Trigger: A downstream server sends exemplarspb.ExemplarsResponse{} or &ExemplarsResponse{Result: nil} over the stream, causing srv.Send to hit `res.GetData() == nil` and return errors.New("empty exemplars data").
Common situations: Buggy or version-mismatched store implementations that send empty responses, middleware altering responses and dropping the Result field, or tests constructing responses by hand and forgetting to call NewExemplarsResponse.
Related errors
AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07).
Data as JSON: /api/errors/03d6bbe2b5c7ba97.
Report an issue: GitHub.
Appendix: source
Thrown at pkg/exemplars/exemplars.go:54
// This field just exist to pseudo-implement the unused methods of the interface.
exemplarspb.Exemplars_ExemplarsServer
ctx context.Context
warnings annotations.Annotations
data []*exemplarspb.ExemplarData
mu sync.Mutex
}
func (srv *exemplarsServer) Send(res *exemplarspb.ExemplarsResponse) error {
if res.GetWarning() != "" {
srv.mu.Lock()
defer srv.mu.Unlock()
srv.warnings.Add(errors.New(res.GetWarning()))
return nil
}
if res.GetData() == nil {
return errors.New("empty exemplars data")
}
srv.mu.Lock()
defer srv.mu.Unlock()
srv.data = append(srv.data, res.GetData())
return nil
}
func (srv *exemplarsServer) Context() context.Context {
return srv.ctx
}
func NewGRPCClient(es exemplarspb.ExemplarsServer) *GRPCClient {
return NewGRPCClientWithDedup(es, nil)
}
func NewGRPCClientWithDedup(es exemplarspb.ExemplarsServer, replicaLabels []string) *GRPCClient {
c := &GRPCClient{View on GitHub (pinned to 35b8b99117)