thanos-io/thanos · error

unknown error while processing Series

Error message

unknown error while processing Series: %v

What it means

The recoverableStoreServer panic-recovery handler caught a panic in the Series RPC handler whose value is not a runtime.Error (e.g. a nil-pointer dereference wrapped value or an explicit panic from a bug). Instead of crashing the server it is downgraded to a warning sent to the client; the client sees a partial result with this warning.

Solutions

  1. Locate the panic via the server logs/stacktrace (the runtime.Error case logs a stack)
  2. Report or fix the underlying bug in the Series serving path
  3. Restart or drain the affected store-gateway if it repeatedly panics
  4. Check the panic is not caused by corrupted block data (run bucket verify)
Defensive patterns

Strategy: try-catch

When it happens

Trigger: Thrown at pkg/store/recover.go:48 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/07cce29aad47e9c4. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/recover.go:48

func (r *recoverableStoreServer) recover(srv storepb.Store_SeriesServer) {
	e := recover()
	if e == nil {
		return
	}

	switch err := e.(type) {
	case runtime.Error:
		// Print the stack trace but do not inhibit the running application.
		buf := make([]byte, 64<<10)
		buf = buf[:runtime.Stack(buf, false)]

		level.Error(r.logger).Log("msg", "runtime panic in TSDB Series server", "err", err.Error(), "stacktrace", string(buf))
		if err := srv.Send(storepb.NewWarnSeriesResponse(err)); err != nil {
			level.Error(r.logger).Log("err", err)
		}
	default:
		if err := srv.Send(storepb.NewWarnSeriesResponse(errors.New(fmt.Sprintf("unknown error while processing Series: %v", e)))); err != nil {
			level.Error(r.logger).Log("err", err)
		}
	}
}

View on GitHub (pinned to 35b8b99117)