thanos-io/thanos · error · api.ApiError

service unavailable

Error message

service unavailable

What it means

A readiness sentinel returned by Handler.getStats when !h.isReady(): the receive handler (its TSDB or writer) is still starting up or shutting down, so the stats endpoint cannot serve tenant statistics. It is surfaced as an api.ErrorInternal, mapped to HTTP 500, and is inherently transient.

Solutions

1. Wait for the handler readiness endpoint before querying stats.
2. Check whether the process is in shutdown.
3. Retry once ready; this is transient.

When it happens

Trigger: Thrown at pkg/receive/handler.go:450 when the library encounters an invalid state.

Common situations: See trigger scenarios.

Understand the failure class

• HTTP status errors: handling 4xx and 5xx responses — how to handle 4xx and 5xx responses properly.

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/726331b12eeb0e08. Report an issue: GitHub.

Appendix: source

Thrown at pkg/receive/handler.go:450

func getStatsLimitParameter(r *http.Request) (int, error) {
	statsLimitStr := r.URL.Query().Get(LimitStatsQueryParam)
	if statsLimitStr == "" {
		return DefaultStatsLimit, nil
	}
	statsLimit, err := strconv.ParseInt(statsLimitStr, 10, 0)
	if err != nil {
		return 0, fmt.Errorf("unable to parse '%s' parameter: %w", LimitStatsQueryParam, err)
	}
	if statsLimit > math.MaxInt {
		return 0, fmt.Errorf("'%s' parameter is larger than %d", LimitStatsQueryParam, math.MaxInt)
	}
	return int(statsLimit), nil
}

func (h *Handler) getStats(r *http.Request, statsByLabelName string) ([]api.TenantStats, *api.ApiError) {
	if !h.isReady() {
		return nil, &api.ApiError{Typ: api.ErrorInternal, Err: fmt.Errorf("service unavailable")}
	}

	tenantID := r.Header.Get(h.options.TenantHeader)
	getAllTenantStats := r.FormValue(AllTenantsQueryParam) == "true"
	if getAllTenantStats && tenantID != "" {
		err := fmt.Errorf("using both the %s parameter and the %s header is not supported", AllTenantsQueryParam, h.options.TenantHeader)
		return nil, &api.ApiError{Typ: api.ErrorBadData, Err: err}
	}

	statsLimit, err := getStatsLimitParameter(r)
	if err != nil {
		return nil, &api.ApiError{Typ: api.ErrorBadData, Err: err}
	}

	if getAllTenantStats {
		return h.options.TSDBStats.TenantStats(statsLimit, statsByLabelName), nil
	}

View on GitHub (pinned to 35b8b99117)