oracle/graal · error · RuleParseError

Unexpected tokens :${rule.substring(m.end(), m.regionEnd())}

Error message

Unexpected tokens :${rule.substring(m.end(), m.regionEnd())}

What it means

Thrown by the RuleParser inside MatchProcessor (the annotation processor for @MatchRule) when tokenizing a match rule leaves a suffix the tokenizer regex cannot consume. Match rules are written as S-expression-like patterns (e.g. 'Add(x, y)'), and after repeatedly applying the token regex the parser verifies it reached the end of the string; leftover characters mean illegal syntax at that position.

Source

Thrown at compiler/src/jdk.graal.compiler.processor/src/jdk/graal/compiler/core/match/processor/MatchProcessor.java:126

        private int current;

        private MatchDescriptor matchDescriptor;

        private final Set<Element> originatingElements = new LinkedHashSet<>();

        private Set<String> requiredPackages = new LinkedHashSet<>();

        RuleParser(String rule) {
            Matcher m = tokenizer.matcher(rule);
            List<String> list = new ArrayList<>();
            int end = 0;
            while (m.lookingAt()) {
                list.add(m.group(1));
                end = m.end();
                m.region(m.end(), m.regionEnd());
            }
            if (end != m.regionEnd()) {
                throw new RuleParseError("Unexpected tokens :" + rule.substring(m.end(), m.regionEnd()));
            }
            tokens = list.toArray(new String[0]);

            matchDescriptor = parseExpression();
            if (!done()) {
                throw new RuleParseError("didn't consume all tokens");
            }
            capturedNames.add(0, "root");
            capturedTypes.add(0, matchDescriptor.nodeType);
        }

        String next() {
            return tokens[current++];
        }

        String peek(String name) {
            if (current >= tokens.length) {
                if (name == null) {

View on GitHub (pinned to a66e9ccd1d)

Solutions

Look at the substring after 'Unexpected tokens :' in the message and delete/fix exactly that part of the @MatchRule value.
Compare against a known-good rule in the same file for the accepted S-expression grammar (identifiers, parentheses, '=' for captured names).
Move complex conditions that the DSL cannot express into the generated match method body instead of the pattern string.

Example fix

// before
@MatchRule("Add(x, y) #")

// after
@MatchRule("Add(x, y)")

Defensive patterns

Strategy: try-catch

Try / catch

RuleParseError and its siblings are compile-time annotation-processing failures; there is no runtime catch site. Wrap javac/mx output parsing in CI to fail fast and echo the offending @MatchRule string.

Prevention

Keep @MatchRule strings on one line; run a build after every rule edit.
Model new rules on existing working ones in the same package.
Cover rule syntax with a small compile-only test module so CI catches bad rules early.

When it happens

Trigger: Writing @MatchRule("Add(x, y) #") or any rule containing characters outside the tokenizer's token set (stray punctuation, comments, unquoted operators) so that after the last matched token the region end exceeds the tokenizer end. The message echoes the unconsumed substring so you can locate it.

Common situations: Hand-editing a @MatchRule string and leaving trailing junk; attempting to write a condition or predicate syntax the match DSL does not support; merge artifacts or stray characters inside the annotation value.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of oracle/graal@a66e9ccd1d (2026-08-14). Data as JSON: /api/errors/c0d4e85f906c49e5. Report an issue: GitHub.