{"record":{"id":"a0739fe1fe9809a2","repo":"thanos-io/thanos","slug":"not-ready-a0739f","errorCode":null,"errorMessage":"Not ready","messagePattern":"Not ready","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"cmd/thanos/rule.go","lineNumber":780,"sourceCode":"\tif tsdbDB != nil {\n\t\ttsdbStore := store.NewTSDBStore(logger, tsdbDB, component.Rule, conf.lset)\n\t\tinfoOptions = append(\n\t\t\tinfoOptions,\n\t\t\tinfo.WithLabelSetFunc(func() []labelpb.ZLabelSet {\n\t\t\t\treturn tsdbStore.LabelSet()\n\t\t\t}),\n\t\t\tinfo.WithStoreInfoFunc(func() (*infopb.StoreInfo, error) {\n\t\t\t\tif httpProbe.IsReady() {\n\t\t\t\t\tmint, maxt := tsdbStore.TimeRange()\n\t\t\t\t\treturn &infopb.StoreInfo{\n\t\t\t\t\t\tMinTime:                      mint,\n\t\t\t\t\t\tMaxTime:                      maxt,\n\t\t\t\t\t\tSupportsSharding:             true,\n\t\t\t\t\t\tSupportsWithoutReplicaLabels: true,\n\t\t\t\t\t\tTsdbInfos:                    tsdbStore.TSDBInfos(),\n\t\t\t\t\t}, nil\n\t\t\t\t}\n\t\t\t\treturn nil, errors.New(\"Not ready\")\n\t\t\t}),\n\t\t\tinfo.WithStatusInfoFunc(),\n\t\t)\n\t\tstoreServer := store.NewLimitedStoreServer(store.NewInstrumentedStoreServer(reg, tsdbStore), reg, conf.storeRateLimits)\n\t\toptions = append(options, grpcserver.WithServer(store.RegisterStoreServer(storeServer, logger)))\n\n\t\t// Add Status server for TSDB statistics\n\t\tstatusSrv := status.NewServer(\n\t\t\tcomponent.Rule.String(),\n\t\t\tstatus.WithTSDBStatisticsGetter(\n\t\t\t\tstatus.TSDBStatisticsGetterFunc(func(limit int, matchers []storepb.LabelMatcher) (map[string]tsdb.Stats, error) {\n\t\t\t\t\tif !httpProbe.IsReady() {\n\t\t\t\t\t\treturn nil, errors.New(\"not ready\")\n\t\t\t\t\t}\n\n\t\t\t\t\t// Check if external labels match the provided matchers.\n\t\t\t\t\textLabels := conf.lset\n\t\t\t\t\tpromMatchers, err := storepb.MatchersToPromMatchers(matchers...)","sourceCodeStart":762,"sourceCodeEnd":798,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/cmd/thanos/rule.go#L762-L798","documentation":"The rule component's Rules/Store gRPC service builds its TSDB store lazily; the info provider callback returns errors.New(\"Not ready\") when httpProbe.IsReady() is false, i.e. the component has finished registering but is not yet fully initialized (TSDB/store not yet set up). Clients querying the store endpoint during this window receive this error.","triggerScenarios":"A store/rules gRPC request (e.g. Info or Series) hits the rule component between process start and readiness: the info provider func runs while initialization (TSDB open, shipping setup) is still in progress.","commonSituations":"Prometheus/Querier scraping store API info immediately after the pod starts; load balancer sending traffic before the readiness probe flips; service discovery registering the endpoint before IsReady becomes true.","solutions":["Retry with backoff on the client side; the error clears once initialization completes.","Gate traffic on the component's HTTP readiness probe (-/-ready) before sending store requests.","Check startup logs for a hung initialization step (e.g. bucket verification, TSDB open) if the state persists.","Configure scrape/discovery refresh intervals to avoid hammering during startup."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// gate traffic on readiness first:\nuntil curl -sf http://rule:10902/-/ready; do sleep 1; done","typeGuard":null,"tryCatchPattern":"// retry with backoff\nfor i := 0; i < 10; i++ {\n  resp, err := client.Info(ctx, req)\n  if err == nil || !strings.Contains(err.Error(), \"Not ready\") { return resp, err }\n  time.Sleep(backoff(i))\n}","preventionTips":["Honor Kubernetes readiness gates before scraping store APIs","Use client-side backoff for store Info/Series calls","Alert on prolonged not-ready state rather than transient errors"],"tags":["grpc","readiness","startup","store"],"backgroundTag":"service-not-ready","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}