oracle/graal · error · RuleParseError

didn't consume all tokens

Error message

didn't consume all tokens

What it means

Thrown by RuleParser in MatchProcessor after tokenizing succeeded but the recursive-descent parseExpression() stopped before consuming all tokens. It means the front of the rule parsed as a complete expression, yet extra tokens remain — typically a top-level parenthesis closed too early or a second pattern where only one is allowed.

Source

Thrown at compiler/src/jdk.graal.compiler.processor/src/jdk/graal/compiler/core/match/processor/MatchProcessor.java:132

        private Set<String> requiredPackages = new LinkedHashSet<>();

        RuleParser(String rule) {
            Matcher m = tokenizer.matcher(rule);
            List<String> list = new ArrayList<>();
            int end = 0;
            while (m.lookingAt()) {
                list.add(m.group(1));
                end = m.end();
                m.region(m.end(), m.regionEnd());
            }
            if (end != m.regionEnd()) {
                throw new RuleParseError("Unexpected tokens :" + rule.substring(m.end(), m.regionEnd()));
            }
            tokens = list.toArray(new String[0]);

            matchDescriptor = parseExpression();
            if (!done()) {
                throw new RuleParseError("didn't consume all tokens");
            }
            capturedNames.add(0, "root");
            capturedTypes.add(0, matchDescriptor.nodeType);
        }

        String next() {
            return tokens[current++];
        }

        String peek(String name) {
            if (current >= tokens.length) {
                if (name == null) {
                    throw new RuleParseError("Out of tokens");
                }
                throw new RuleParseError("Out of tokens looking for %s", name);
            }
            return tokens[current];
        }

View on GitHub (pinned to a66e9ccd1d)

Solutions

1. Wrap the leftover tokens into the existing expression (add the missing '(' earlier) or remove them.
2. Re-read the rule as a tree: exactly one root node whose inputs fill its parentheses.
3. Split genuine multi-pattern matching into separate @MatchRule methods.

Example fix

// before
@MatchRule("(Add(x)) (Constant)")

// after
@MatchRule("Add(x, Constant)")

No runtime catch: this is a RuleParseError raised inside javac annotation processing. Treat the compiler error as the signal; CI should treat any 'RuleParseError' string in build output as a hard failure.

When it happens

Trigger: A @MatchRule value like "Add(x) y" where 'Add(x)' completes and 'y' is left over, or "(Add(x)) (Constant)" — two expressions at top level. done() returns current == tokens.length; anything short of that triggers this RuleParseError.

Common situations: Nesting mistakes when composing complex match patterns; forgetting that only a single top-level node expression is permitted; refactoring a rule and leaving an orphaned sub-pattern.

Related errors

• Unexpected tokens :${rule.substring(m.end(), m.regionEnd())}
• Out of tokens
• Out of tokens looking for %s
• not enough inputs for ${descriptor.name}
• Too many arguments to ${descriptor.nodeType.nodeClass}

AI-assisted analysis of oracle/graal@a66e9ccd1d (2026-08-14). Data as JSON: /api/errors/ca6af8de65c2cf48. Report an issue: GitHub.