What causes "invalid regular expression in label_replace(): %s" in prometheus/prometheus?

Fix the regex to valid RE2/Go syntax (test with Go's regexp or an RE2 tester; remove lookaheads/backreferences)

invalid regular expression in label_replace(): %s

Error message

invalid regular expression in label_replace(): %s

What it means

evalLabelReplace compiles the regular expression wrapped as ^(?s:<regex>)$ before evaluating the inner vector. If the user-supplied regex in label_replace(v, dst, repl, src, regex) does not compile, the evaluator panics with 'invalid regular expression in label_replace(): <regex>'; the query engine recovers the panic and returns it as a query error at the offending position.

Source

Thrown at promql/functions.go:2493

		}
		prevSample = curSample
	}

	return append(enh.Out, Sample{F: float64(changes)}), nil
}

// label_replace function operates only on series; does not look at timestamps or values.
func (ev *evaluator) evalLabelReplace(ctx context.Context, args parser.Expressions) (parser.Value, annotations.Annotations) {
	var (
		dst      = stringFromArg(args[1])
		repl     = stringFromArg(args[2])
		src      = stringFromArg(args[3])
		regexStr = stringFromArg(args[4])
	)

	regex, err := regexp.Compile("^(?s:" + regexStr + ")$")
	if err != nil {
		panic(fmt.Errorf("invalid regular expression in label_replace(): %s", regexStr))
	}
	if !model.UTF8Validation.IsValidLabelName(dst) {
		panic(fmt.Errorf("invalid destination label name in label_replace(): %s", dst))
	}

	val, ws := ev.eval(ctx, args[0])
	matrix := val.(Matrix)
	lb := labels.NewBuilder(labels.EmptyLabels())

	for i, el := range matrix {
		srcVal := el.Metric.Get(src)
		indexes := regex.FindStringSubmatchIndex(srcVal)
		if indexes != nil { // Only replace when regexp matches.
			res := regex.ExpandString([]byte{}, repl, srcVal, indexes)
			lb.Reset(el.Metric)
			lb.Set(dst, string(res))
			matrix[i].Metric = lb.Labels()
			if dst == model.MetricNameLabel {

View on GitHub (pinned to 44d6a0e0b1)

Solutions

Fix the regex to valid RE2/Go syntax (test with Go's regexp or an RE2 tester; remove lookaheads/backreferences)
Mind YAML escaping in rules files: use single quotes or double the backslashes appropriately
If the regex is dynamic, sanitize/validate it before injecting into the query
Test with a minimal query in the UI expression browser first

Example fix

# before (PCRE lookahead, invalid in RE2)
label_replace(up, "d", "$1", "instance", "([^:]+)(?=:.*)")

# after
label_replace(up, "d", "$1", "instance", "([^:]+):.*")

Defensive patterns

Strategy: validation

Validate before calling

// Go/RE2 pre-check before sending the query
if _, err := regexp.Compile("^(?s:" + regexStr + ")$"); err != nil { return fmt.Errorf("bad regex: %w", err); }

Prevention

Test label_replace regexes with Go's regexp package (RE2): no lookaheads, no backreferences
Double-check escaping when embedding regex in YAML rules
Validate dynamically assembled regexes before query submission

When it happens

Trigger: Calling label_replace with a syntactically invalid Go regexp: unbalanced parentheses, invalid escapes like '\d' (Go uses \d but not \z style PCRE syntax such as lookaheads (?=...)), or stray repetition like '*', e.g. label_replace(up, "a", "$1", "job", "(.*)extra)").

Common situations: Porting regexes from PCRE/Python/JS (lookaheads, backreferences not supported by RE2); dynamic regex built from label values containing metacharacters; escaping mistakes when embedding regex in YAML rules (double-escaping issues).

Related errors

AI-assisted analysis of prometheus/prometheus@44d6a0e0b1 (2026-08-15). Data as JSON: /api/errors/bd7aa7b37b7bf29f. Report an issue: GitHub.