{"record":{"id":"4022d11e54dff79e","repo":"thanos-io/thanos","slug":"hashring-has-changed-server-is-not-ready-to-recei","errorCode":null,"errorMessage":"hashring has changed; server is not ready to receive requests","messagePattern":"hashring has changed; server is not ready to receive requests","errorType":"http","errorClass":null,"httpStatus":503,"severity":"error","filePath":"cmd/thanos/receive.go","lineNumber":763,"sourceCode":"\t\t\tselect {\n\t\t\tcase <-cancel:\n\t\t\t\treturn nil\n\t\t\tcase _, ok := <-hashringChangedChan:\n\t\t\t\tif !ok {\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\n\t\t\t\t// When using Ketama as the hashring algorithm, there is no need to flush the TSDB head.\n\t\t\t\t// If new receivers were added to the hashring, existing receivers will not need to\n\t\t\t\t// ingest additional series.\n\t\t\t\t// If receivers are removed from the hashring, existing receivers will only need\n\t\t\t\t// to ingest a subset of the series that were assigned to the removed receivers.\n\t\t\t\t// As a result, changing the hashring produces no churn, hence no need to force\n\t\t\t\t// head compaction and upload.\n\t\t\t\tflushHead := !initialized || hashringAlgorithm != receive.AlgorithmKetama\n\t\t\t\tif flushHead {\n\t\t\t\t\tmsg := \"hashring has changed; server is not ready to receive requests\"\n\t\t\t\t\tstatusProber.NotReady(errors.New(msg))\n\t\t\t\t\tlevel.Info(logger).Log(\"msg\", msg)\n\n\t\t\t\t\tlevel.Info(logger).Log(\"msg\", \"updating storage\")\n\t\t\t\t\tdbUpdatesStarted.Inc()\n\t\t\t\t\tif err := dbs.Flush(); err != nil {\n\t\t\t\t\t\treturn errors.Wrap(err, \"flushing storage\")\n\t\t\t\t\t}\n\t\t\t\t\tif err := dbs.Open(); err != nil {\n\t\t\t\t\t\treturn errors.Wrap(err, \"opening storage\")\n\t\t\t\t\t}\n\t\t\t\t\tif upload {\n\t\t\t\t\t\tuploadC <- struct{}{}\n\t\t\t\t\t\t<-uploadDone\n\t\t\t\t\t}\n\t\t\t\t\tdbUpdatesCompleted.Inc()\n\t\t\t\t\tstatusProber.Ready()\n\t\t\t\t\tlevel.Info(logger).Log(\"msg\", \"storage started, and server is ready to receive requests\")\n\t\t\t\t\tdbUpdatesCompleted.Inc()","sourceCodeStart":745,"sourceCodeEnd":781,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/cmd/thanos/receive.go#L745-L781","documentation":"Not an error object but a readiness message: when the hashring changes, the server is marked not-ready and this message is logged/probed while head flushing and reopening complete. It fires on every hashring update (unless Ketama with no flush needed).","triggerScenarios":"Thrown at cmd/thanos/receive.go:763 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Wait for flush/reopen to complete; readiness restores automatically","During frequent hashring churn consider Ketama to reduce flushes","Scale replication changes gradually"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}