elastic/elasticsearch · error · IllegalArgumentException

Can not convert grok patterns to regular expression

Error message

Can not convert grok patterns to regular expression

What it means

Thrown by Grok.toRegex() when the pattern expansion loop exceeds MAX_TO_REGEX_ITERATIONS (100,000 iterations) without converging to a fully-expanded regex. Each iteration resolves one %{PATTERN} reference. This is a safety valve against pathological patterns that expand exponentially or have indirect circular references (A→B→A) that are not caught by the direct self-reference check.

Source

Thrown at libs/grok/src/main/java/org/elasticsearch/grok/Grok.java:171

            String grokPart;
            if (namedCaptures && subName != null) {
                grokPart = String.format(Locale.US, "(?<%s>%s)", namedPatternRef, pattern);
            } else if (namedCaptures) {
                grokPart = String.format(Locale.US, "(?:%s)", pattern);
            } else {
                grokPart = String.format(Locale.US, "(?<%s>%s)", patternName + "_" + result, pattern);
            }
            String start = new String(grokPatternBytes, 0, result, StandardCharsets.UTF_8);
            String rest = new String(
                grokPatternBytes,
                region.getEnd(0),
                grokPatternBytes.length - region.getEnd(0),
                StandardCharsets.UTF_8
            );
            grokPattern = grokPart + rest;
            res.append(start);
        }
        throw new IllegalArgumentException("Can not convert grok patterns to regular expression");
    }

    /**
     * Checks whether a specific text matches the defined grok expression.
     *
     * @param text the string to match
     * @return true if grok expression matches text or there is a timeout, false otherwise.
     */
    public boolean match(String text) {
        Matcher matcher = compiledExpression.matcher(text.getBytes(StandardCharsets.UTF_8));
        int result;
        try {
            matcherWatchdog.register(matcher);
            result = matcher.search(0, text.length(), Option.DEFAULT);
        } finally {
            matcherWatchdog.unregister(matcher);
        }
        handleInterrupted(result);

View on GitHub (pinned to db6a809a66)

Solutions

1. Check for indirect circular references in custom pattern definitions (A→B→A patterns).
2. Simplify the grok pattern to reduce the depth of pattern nesting.
3. Flatten frequently-referenced patterns inline to reduce expansion depth.
4. Audit custom pattern files for chains where patterns reference each other in a cycle.

Example fix

// before — indirect circular reference
// A %{B}
// B %{A}

// after — break the cycle
// A \d+
// B %{A}

// Detect indirect circular references by building a dependency graph
Map<String, Set<String>> deps = buildPatternDependencyGraph(patternBank);
for (String pattern : deps.keySet()) {
    if (hasCycle(pattern, deps, new HashSet<>())) {
        throw new IllegalArgumentException("Indirect circular reference detected involving pattern: " + pattern);
    }
}

try {
    new Grok(bank, pattern, callback);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("Can not convert")) {
        // pattern expansion too deep — check for indirect cycles or simplify nesting
    }
    throw e;
}

When it happens

Trigger: Constructing a Grok object with a pattern that causes extremely deep or non-terminating expansion. This can happen with: indirect circular references (A references B, B references A — the direct check only catches A→A); deeply nested pattern chains; patterns that expand to extremely large regexes through combinatorial expansion.

Common situations: Custom grok pattern with indirect circular references (not caught by the direct self-reference check); importing a large external pattern set with complex interdependencies that create exponential expansion; a pattern chain that is legitimately very deep (exceeding 100K references).

Related errors

• circular reference in pattern back [{}]
• Unable to find pattern [{}] in Grok's pattern dictionary
• unsupported ECS compatibility mode [{}]
• grok pattern matching was interrupted after [{}] ms
• failed to load built-in patterns

AI-assisted analysis of elastic/elasticsearch@db6a809a66 (2026-08-12). Data as JSON: /api/errors/914050d58b9b645c. Report an issue: GitHub.