thanos-io/thanos · warning

Not ready

Error message

Not ready

What it means

Receive's gRPC Info server provides a TSDB info function that checks whether the process is ready via httpProbe.IsReady(). If not ready, the RPC returns errors.New("Not ready"). This is a transient readiness error served over gRPC while the receive component is still starting up (TSDBs not yet opened/registered) or shutting down.

Solutions

  1. Retry the Info request with backoff; the error clears once the instance becomes ready.
  2. Ensure clients (querier) exclude unready endpoints via readiness probes in Service/endpoints selection.
  3. Check why readiness is delayed: TSDB opening time, blocked init, or failing /-/ready endpoint.
  4. If it persists, inspect receive logs for startup failures blocking readiness.
Defensive patterns

Strategy: retry

Validate before calling

resp, err := checkReady(ctx, client) // GET /-/ready returns 200 only when ready

Try / catch

var info *storepb.InfoResponse
backoff := 100 * time.Millisecond
for i := 0; i < 10; i++ {
  info, err = client.Info(ctx, req)
  if err == nil || !strings.Contains(err.Error(), "Not ready") { break }
  time.Sleep(backoff)
  backoff *= 2
}

Prevention

When it happens

Trigger: Calling the Info gRPC API on a thanos receive instance whose HTTP readiness probe has not yet reported ready — during startup, TSDB opening, or shutdown — from a querier/store client issuing Info requests.

Common situations: Querier discovery polls a freshly started receive pod before its readiness gate passes; load balancer sends Info RPCs to a not-yet-ready replica; extended TSDB head opening makes readiness take long.

Related errors

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/e4de4e4df0e07875. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/receive.go:405

			StoreServer:          mts,
			WriteableStoreServer: webHandler,
		}

		infoSrv := info.NewInfoServer(
			component.Receive.String(),
			info.WithLabelSetFunc(func() []labelpb.ZLabelSet { return proxy.LabelSet() }),
			info.WithStoreInfoFunc(func() (*infopb.StoreInfo, error) {
				if httpProbe.IsReady() {
					minTime, maxTime := proxy.TimeRange()
					return &infopb.StoreInfo{
						MinTime:                      minTime,
						MaxTime:                      maxTime,
						SupportsSharding:             true,
						SupportsWithoutReplicaLabels: true,
						TsdbInfos:                    proxy.TSDBInfos(),
					}, nil
				}
				return nil, errors.New("Not ready")
			}),
			info.WithExemplarsInfoFunc(),
			info.WithStatusInfoFunc(),
		)

		statusSrv := status.NewServer(
			component.Receive.String(),
			status.WithTSDBStatisticsGetter(
				status.TSDBStatisticsGetterFunc(func(limit int, matchers []storepb.LabelMatcher) (map[string]tsdb.Stats, error) {
					if !httpProbe.IsReady() {
						return nil, errors.New("not ready")
					}

					promMatchers, err := storepb.MatchersToPromMatchers(matchers...)
					if err != nil {
						return nil, errors.Wrap(err, "failed to convert matchers")
					}

View on GitHub (pinned to 35b8b99117)