thanos-io/thanos · error · LimitError

The ingester.max-global-series-per-user limit is…

Error message

The ingester.max-global-series-per-user limit is unsupported if distributor.shard-by-all-labels is disabled

What it means

errMaxGlobalSeriesPerUserValidation enforces a configuration invariant in Limits.Validate: MaxGlobalSeriesPerUser (ingester.max-global-series-per-user) is a per-cluster aggregate limit that only makes sense when series are consistently sharded across ingesters, i.e. when distributor.shard-by-all-labels is enabled. Setting a positive global limit with sharding disabled is rejected at startup.

Solutions

Set distributor.shard-by-all-labels: true in the config.
Or remove/zero ingester.max-global-series-per-user and rely on ingester.max-series-per-user (per-instance) limits instead.
Note the migration requirement: enabling shard-by-all-labels requires ingester ring resharding — plan a rollout per upstream docs.
Run config validation locally before deploy (Validate is invoked during flag parsing/config load) to catch the mismatch early.

Example fix

// before (config)
ingester:
  max-global-series-per-user: 100000
# after (config)
distributor:
  shard-by-all-labels: true
ingester:
  max-global-series-per-user: 100000

Defensive patterns

Strategy: validation

Validate before calling

if limits.MaxGlobalSeriesPerUser > 0 && !shardByAllLabels {
    return errors.New("max-global-series-per-user requires shard-by-all-labels=true")
}
// then call Limits.Validate(...)

Try / catch

if err := limits.Validate(shardByAllLabels); err != nil {
    return fmt.Errorf("invalid limits config: %w", err)
}

Prevention

Validate config in CI with Limits.Validate before deploy.
Pair any max-global-series-per-user setting with shard-by-all-labels: true.
Document the interdependency in your config templates.
Prefer per-instance limits if shard-by-all-labels cannot be enabled.

When it happens

Trigger: Limits.Validate is called (config load / startup) with ingester.max-global-series-per-user > 0 while distributor.shard-by-all-labels is false; also asserted in limits tests.

Common situations: Operator copies a config between Cortex/Thanos deployments and enables the global limit but forgets shard-by-all-labels; migration from old default (shard-by-all-labels=false) to newer configs that assume it is on.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/78e13812ea577271. Report an issue: GitHub.

Appendix: source

Thrown at internal/cortex/util/validation/limits.go:23

import (
	"bytes"
	"encoding/json"
	"errors"
	"flag"
	"maps"
	"math"
	"strings"
	"time"

	"github.com/prometheus/common/model"
	"github.com/prometheus/prometheus/model/relabel"
	"golang.org/x/time/rate"

	"github.com/thanos-io/thanos/internal/cortex/util/flagext"
)

var errMaxGlobalSeriesPerUserValidation = errors.New("The ingester.max-global-series-per-user limit is unsupported if distributor.shard-by-all-labels is disabled")

// Supported values for enum limits
const (
	LocalIngestionRateStrategy  = "local"
	GlobalIngestionRateStrategy = "global"
)

// LimitError are errors that do not comply with the limits specified.
type LimitError string

func (e LimitError) Error() string {
	return string(e)
}

// Limits describe all the limits for users; can be used to describe global default
// limits via flags, or per-user limits via yaml config.
type Limits struct {
	// Distributor enforced limits.

View on GitHub (pinned to 35b8b99117)