thanos-io/thanos · error

toPostingGroup

Error message

toPostingGroup

What it means

In matchersToPostingGroups, matchers with multiple values (e.g. from regex expansion or set matchers) are merged, and each value is converted into a posting group via toPostingGroup, which resolves label values through the provided lvalsFunc (possibly hitting the index/bucket). Any per-value failure is wrapped as 'toPostingGroup'. It means a single matcher value could not be turned into a posting group fetch.

Solutions

1. Inspect the wrapped inner error to distinguish storage failure from index corruption.
2. Retry the query for transient object-store errors.
3. Refresh store-gateway index-header caches/reload blocks.
4. Rewrite the matcher with explicit values (foo in {a,b,c}) to avoid expensive value expansion.

// ensure lvalsFunc never returns error for cached names
lvalsFunc := func(name string) ([]string, error) {
	if v, ok := cache[name]; ok { return v, nil }
	return indexHeader.LabelValues(ctx, name)
}

if err != nil {
	if isTransient(err) { /* retry value expansion */ }
	return nil, errors.Wrap(err, "toPostingGroup")
}

When it happens

Trigger: toPostingGroup(ctx, lvalsFunc, val) errors while expanding one value of a multi-valued matcher — the label-values lookup fails (index header read error, lazy bucket fetch failure).

Common situations: Queries using regex/set matchers (foo=~"a|b|c") where resolving candidate values requires bucket-backed label values and storage fails; stale index-header caches; S3 throttling during value expansion.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• fetch and expand postings
• failed to convert matchers
• failed to convert matchers
• failed to convert matchers
• postings

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/7a44d02b974ba417. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/bucket.go:2900

	pgs := make([]*postingGroup, 0, len(matchersMap))
	// NOTE: Derived from tsdb.PostingsForMatchers.
	for _, values := range matchersMap {
		var (
			mergedPG     *postingGroup
			pg           *postingGroup
			vals         []string
			err          error
			valuesCached bool
		)
		lvalsFunc := lvalsFn
		matchers := make([]*labels.Matcher, 0, len(vals))
		// Merge PostingGroups with the same matcher into 1 to
		// avoid fetching duplicate postings.
		for _, val := range values {
			pg, vals, err = toPostingGroup(ctx, lvalsFunc, val)
			if err != nil {
				return nil, errors.Wrap(err, "toPostingGroup")
			}
			// Cache label values because label name is the same.
			if !valuesCached && vals != nil {
				lvals := vals
				lvalsFunc = func(_ string) ([]string, error) {
					return lvals, nil
				}
				valuesCached = true
			}

			// If this groups adds nothing, it's an empty group. We can shortcut this, since intersection with empty
			// postings would return no postings anyway.
			// E.g. label="non-existing-value" returns empty group.
			if !pg.addAll && len(pg.addKeys) == 0 {
				return nil, nil
			}
			if mergedPG == nil {
				mergedPG = pg

View on GitHub (pinned to 35b8b99117)