thanos-io/thanos · info

negative offset

Error message

negative offset

What it means

resultsCache.isOffsetCachable raises this sentinel when a query contains a negative `offset`, because cached results for negative offsets would depend on data newer than the cached window and would be wrong. The cache skips such responses.

Solutions

1. Rewrite the query to use a positive offset or shift the query range instead
2. Accept no caching for this query (behavior is correct, just uncached)
3. Fetch the comparison data in a separate time-shifted query

Example fix

// before
metric offset -5m
// after: run query with end time 5m earlier, or positive offset
metric offset 5m

if strings.Contains(query, "offset -") {
	return errors.New("negative offsets bypass results cache")
}

resp, err := frontend.Query(ctx, q)
if err != nil && strings.Contains(err.Error(), "negative offset") {
	// rewrite query or skip caching path
}

When it happens

Trigger: Querying with PromQL containing `offset -5m` (or any negative offset) while results caching is enabled; detected via extpromql.ParseExpr during shouldCacheResponse.

Common situations: Dashboards comparing current vs. past values using negative offsets (a common Grafana idiom that is unsafe with result caching); users porting queries that relied on future data.

Related errors

• at modifier after end
• use of multiple cache storage systems is not supported
• unsupported compression type
• bad cached type
• querier.cache-results may only be enabled in conjunction…

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/b0e9e8581e16752d. Report an issue: GitHub.

Appendix: source

Thrown at internal/cortex/querier/queryrange/results_cache.go:427

				atModCachable = false
				return errAtModifierAfterEnd
			}
		case *parser.SubqueryExpr:
			if e.Timestamp != nil && (*e.Timestamp > end || *e.Timestamp > maxCacheTime) {
				atModCachable = false
				return errAtModifierAfterEnd
			}
		}
		return nil
	})

	return atModCachable
}

// isOffsetCachable returns true if the offset is positive, result is safe to cache.
// and false when offset is negative, result is not cached.
func (s resultsCache) isOffsetCachable(r Request) bool {
	var errNegativeOffset = errors.New("negative offset")
	query := r.GetQuery()
	if !strings.Contains(query, "offset") {
		return true
	}
	expr, err := extpromql.ParseExpr(query)
	if err != nil {
		level.Warn(s.logger).Log("msg", "failed to parse query, considering offset as not cachable", "query", query, "err", err)
		return false
	}

	offsetCachable := true
	parser.Inspect(expr, func(n parser.Node, _ []parser.Node) error {
		switch e := n.(type) {
		case *parser.VectorSelector:
			if e.OriginalOffset < 0 {
				offsetCachable = false
				return errNegativeOffset
			}

View on GitHub (pinned to 35b8b99117)