thanos-io/thanos · warning

not ready

Error message

not ready

What it means

The Status gRPC server's TSDB statistics getter returns errors.New("not ready") when the sidecar's HTTP probe indicates it is not ready. Requests for TSDB stats (cardinality, posting) are rejected until the sidecar finished startup validation against Prometheus.

Solutions

  1. Retry the request after the sidecar's /-/ready endpoint returns success
  2. Add readiness gating in clients/UI before issuing TSDB stats queries
  3. Investigate why the sidecar is not ready if the condition persists (check startup logs)
  4. Restart the sidecar once Prometheus is confirmed healthy

Example fix

// before
stats, err := statusClient.TSDBStatistics(ctx, req)   // during restart
// after
if err := waitForReady(sidecarAddr, 60*time.Second); err != nil {
    return err
}
stats, err := statusClient.TSDBStatistics(ctx, req)
Defensive patterns

Strategy: retry

Validate before calling

if resp, err := http.Get(sidecarURL + "/-/ready"); err != nil || resp.StatusCode != 200 {
    return errors.New("sidecar not ready; defer TSDB statistics request")
}

Try / catch

err := retry.Do(3, func() error {
    _, err := statusClient.TSDBStatistics(ctx, req)
    return err
})

Prevention

When it happens

Trigger: status.NewServer TSDBStatisticsGetterFunc invoked while !httpProbe.IsReady() — sidecar still starting (labels/version not yet loaded) or after readiness was revoked.

Common situations: Users hitting the Thanos UI/Status TSDB stats page during a sidecar restart; automated tooling calling status TSDBStatistics RPC at boot; probes racing the startup sequence.

Related errors

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/edea5c19b9160700. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/sidecar.go:361

						SupportsWithoutReplicaLabels: true,
						TsdbInfos:                    promStore.TSDBInfos(),
					}, nil
				}
				return nil, errors.New("Not ready")
			}),
			info.WithExemplarsInfoFunc(),
			info.WithRulesInfoFunc(),
			info.WithTargetsInfoFunc(),
			info.WithMetricMetadataInfoFunc(),
			info.WithStatusInfoFunc(),
		)

		statusSrv := status.NewServer(
			component.Sidecar.String(),
			status.WithTSDBStatisticsGetter(
				status.TSDBStatisticsGetterFunc(func(limit int, matchers []storepb.LabelMatcher) (map[string]tsdb.Stats, error) {
					if !httpProbe.IsReady() {
						return nil, errors.New("not ready")
					}

					ctx, cancel := context.WithTimeout(context.Background(), conf.prometheus.getConfigTimeout)
					defer cancel()

					// Check if external labels match the provided matchers.
					extLabels := m.Labels()
					promMatchers, err := storepb.MatchersToPromMatchers(matchers...)
					if err != nil {
						return nil, errors.Wrap(err, "failed to convert matchers")
					}
					for _, matcher := range promMatchers {
						if !matcher.Matches(extLabels.Get(matcher.Name)) {
							// External labels don't match, return empty result.
							return nil, nil
						}
					}

View on GitHub (pinned to 35b8b99117)