thanos-io/thanos · error

opening storage

Error message

opening storage

What it means

Reopening the MultiTSDB storage after a hashring-change flush failed. The tenant DBs cannot be opened (locks, corruption, permissions), so the server stays not-ready and the run group receives the error.

Solutions

1. Inspect underlying TSDB open error in logs
2. Check for corrupt blocks in tenant dirs
3. Verify no concurrent process holds TSDB locks

When it happens

Trigger: Thrown at cmd/thanos/receive.go:772 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/77a4422595341cac. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/receive.go:772

				// If new receivers were added to the hashring, existing receivers will not need to
				// ingest additional series.
				// If receivers are removed from the hashring, existing receivers will only need
				// to ingest a subset of the series that were assigned to the removed receivers.
				// As a result, changing the hashring produces no churn, hence no need to force
				// head compaction and upload.
				flushHead := !initialized || hashringAlgorithm != receive.AlgorithmKetama
				if flushHead {
					msg := "hashring has changed; server is not ready to receive requests"
					statusProber.NotReady(errors.New(msg))
					level.Info(logger).Log("msg", msg)

					level.Info(logger).Log("msg", "updating storage")
					dbUpdatesStarted.Inc()
					if err := dbs.Flush(); err != nil {
						return errors.Wrap(err, "flushing storage")
					}
					if err := dbs.Open(); err != nil {
						return errors.Wrap(err, "opening storage")
					}
					if upload {
						uploadC <- struct{}{}
						<-uploadDone
					}
					dbUpdatesCompleted.Inc()
					statusProber.Ready()
					level.Info(logger).Log("msg", "storage started, and server is ready to receive requests")
					dbUpdatesCompleted.Inc()
				}
				initialized = true
			}
		}
	}, func(err error) {
		close(cancel)
	})

	if upload {

View on GitHub (pinned to 35b8b99117)