antlr/antlr4 · error · UnsupportedOperationException

Precedence predicates are not supported in lexers.

Error message

Precedence predicates are not supported in lexers.

What it means

LexerATNSimulator.closure hits a PrecedencePredicateTransition while computing lexer DFA reach. Precedence predicates exist only for parser rules (left-recursive rule precedence, grammar option 'options { contextSuperClass... }' style precedence), and the lexer engine cannot evaluate them. Their presence in a lexer ATN means the grammar/tool produced something the lexer runtime cannot run.

Source

Thrown at runtime/Java/src/org/antlr/v4/runtime/atn/LexerATNSimulator.java:481

	protected LexerATNConfig getEpsilonTarget(CharStream input,
										   LexerATNConfig config,
										   Transition t,
										   ATNConfigSet configs,
										   boolean speculative,
										   boolean treatEofAsEpsilon)
	{
		LexerATNConfig c = null;
		switch (t.getSerializationType()) {
			case Transition.RULE:
				RuleTransition ruleTransition = (RuleTransition)t;
				PredictionContext newContext =
					SingletonPredictionContext.create(config.context, ruleTransition.followState.stateNumber);
				c = new LexerATNConfig(config, t.target, newContext);
				break;

			case Transition.PRECEDENCE:
				throw new UnsupportedOperationException("Precedence predicates are not supported in lexers.");

			case Transition.PREDICATE:
				/*  Track traversing semantic predicates. If we traverse,
				 we cannot add a DFA state for this "reach" computation
				 because the DFA would not test the predicate again in the
				 future. Rather than creating collections of semantic predicates
				 like v3 and testing them on prediction, v4 will test them on the
				 fly all the time using the ATN not the DFA. This is slower but
				 semantically it's not used that often. One of the key elements to
				 this predicate mechanism is not adding DFA states that see
				 predicates immediately afterwards in the ATN. For example,

				 a : ID {p1}? | ID {p2}? ;

				 should create the start state for rule 'a' (to save start state
				 competition), but should not create target of ID state. The
				 collection of ATN states the following ID references includes
				 states reached by traversing predicates. Since this is when we

View on GitHub (pinned to 7d5770395b)

Solutions

1. Remove precedence constructs (left-recursive-style precedence, precedence semantic predicates) from lexer rules; lexers only support plain semantic predicates {...}?
2. Verify you are not constructing a LexerInterpreter from parser .interp data; build a ParserInterpreter for parser grammars
3. Align the ANTLR tool and runtime versions, then regenerate all lexers/parsers

Example fix

// before
// lexer grammar with precedence-style predicate (invalid in lexer)
TOKEN: {precedence > 0}? <= 'a' 'b'+ ;
// after
// use a plain semantic predicate in the lexer rule
TOKEN: {this.inParens()}? 'b'+ ;

// Before building a LexerInterpreter, confirm the serialized ATN has no PRECEDENCE transitions
int[] data = /* serialized ints */;
boolean hasPrecedence = new ATNDeserializer().deserialize(data) // inspect
        .states.stream().filter(Objects::nonNull)
        .flatMap(s -> s.transitions().stream())
        .anyMatch(t -> t.getSerializationType() == Transition.PRECEDENCE);
if (hasPrecedence) throw new IllegalArgumentException("grammar not usable as lexer");

static boolean isLexerSafe(ATN atn) {
    for (ATNState s : atn.states) {
        if (s == null) continue;
        for (int i = 0; i < s.getNumberOfTransitions(); i++) {
            if (s.transition(i) instanceof PrecedencePredicateTransition) return false;
        }
    }
    return true;
}

When it happens

Trigger: A lexer grammar whose generated ATN contains PrecedencePredicate transitions: using precedence operators/semantic predicates of the precedence form ({p}? <= ... is parser-only) in lexer rules, or feeding a parser ATN/serialized data to a lexer interpreter (e.g. building a LexerInterpreter over .interp data generated for a parser). Mismatched tool/runtime versions can also deserialize an ATN with unexpected transition types.

Common situations: Using the Grammar interpreter API (Grammar, LexerInterpreter) with an .interp file produced from a grammar that has left recursion in lexer mode or is actually a combined grammar misinterpreted as a lexer. Downgrading the runtime below the tool version so new ATN features appear in old runtime code paths.

Related errors

• should only be one op per index
• Parser can't discover a lexer to use
• The specified lexer action type %s is not valid.
• The specified lexer action type %s is not valid.
• This ATN simulator does not support clearing the DFA.

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/999435e486939686. Report an issue: GitHub.