thanos-io/thanos · warning

Not ready

Error message

Not ready

What it means

The Info service's TSDBInfos getter returns errors.New("Not ready") when the sidecar's HTTP readiness probe has not yet succeeded. Clients calling the gRPC Info API before the sidecar has validated Prometheus connectivity and loaded labels get this sentinel error instead of TSDB info.

Solutions

1. Wait for the sidecar's HTTP readiness endpoint (/-/ready) to report ready before querying Info API
2. Configure clients (store gateways) to retry on 'Not ready' errors during rollout
3. Gate traffic with Kubernetes readiness probes on the gRPC/HTTP ports
4. If never becomes ready, fix the underlying startup error (see earlier sidecar logs)

Example fix

// before
resp, err := infoClient.Info(ctx, &infoPb.InfoReq{})   // right after pod start
// after
err := wait.PollImmediate(2*time.Second, 2*time.Minute, func() (bool, error) {
    return probeReady(sidecarHTTPAddr) == nil, nil
})
resp, err := infoClient.Info(ctx, &infoPb.InfoReq{})

ready, err := isSidecarReady(sidecarHTTPAddr)
if err != nil || !ready {
    return errors.New("sidecar info API not ready yet, defer Info RPC")
}

err := wait.PollImmediate(2*time.Second, timeout, func() (bool, error) {
    _, err := infoClient.Info(ctx, &infoPb.InfoReq{})
    return err == nil, nil
})

When it happens

Trigger: gRPC info.InfoServer Info/TSDBInfos RPC invoked while httpProbe.IsReady() is false — during startup before external labels were fetched, or after a readiness-affecting failure.

Common situations: Store Gateway/Compactor querying the sidecar immediately after rollout; load balancers probing the Info API before readiness; monitoring scraping info RPCs at boot.

Related errors

• not ready
• building gRPC client
• endpoint
• Not ready
• Not ready

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/33f1d1c20d7c7f60. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/sidecar.go:347

		exemplarSrv := exemplars.NewPrometheus(conf.prometheus.url, c, m.Labels)

		infoSrv := info.NewInfoServer(
			component.Sidecar.String(),
			info.WithLabelSetFunc(func() []labelpb.ZLabelSet {
				return promStore.LabelSet()
			}),
			info.WithStoreInfoFunc(func() (*infopb.StoreInfo, error) {
				if httpProbe.IsReady() {
					mint, maxt := m.Timestamps()
					return &infopb.StoreInfo{
						MinTime:                      mint,
						MaxTime:                      maxt,
						SupportsSharding:             true,
						SupportsWithoutReplicaLabels: true,
						TsdbInfos:                    promStore.TSDBInfos(),
					}, nil
				}
				return nil, errors.New("Not ready")
			}),
			info.WithExemplarsInfoFunc(),
			info.WithRulesInfoFunc(),
			info.WithTargetsInfoFunc(),
			info.WithMetricMetadataInfoFunc(),
			info.WithStatusInfoFunc(),
		)

		statusSrv := status.NewServer(
			component.Sidecar.String(),
			status.WithTSDBStatisticsGetter(
				status.TSDBStatisticsGetterFunc(func(limit int, matchers []storepb.LabelMatcher) (map[string]tsdb.Stats, error) {
					if !httpProbe.IsReady() {
						return nil, errors.New("not ready")
					}

					ctx, cancel := context.WithTimeout(context.Background(), conf.prometheus.getConfigTimeout)
					defer cancel()

View on GitHub (pinned to 35b8b99117)