{"record":{"id":"999435e486939686","repo":"antlr/antlr4","slug":"precedence-predicates-are-not-supported-in-lexers","errorCode":null,"errorMessage":"Precedence predicates are not supported in lexers.","messagePattern":"Precedence predicates are not supported in lexers\\.","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"runtime/Java/src/org/antlr/v4/runtime/atn/LexerATNSimulator.java","lineNumber":481,"sourceCode":"\n\tprotected LexerATNConfig getEpsilonTarget(CharStream input,\n\t\t\t\t\t\t\t\t\t\t   LexerATNConfig config,\n\t\t\t\t\t\t\t\t\t\t   Transition t,\n\t\t\t\t\t\t\t\t\t\t   ATNConfigSet configs,\n\t\t\t\t\t\t\t\t\t\t   boolean speculative,\n\t\t\t\t\t\t\t\t\t\t   boolean treatEofAsEpsilon)\n\t{\n\t\tLexerATNConfig c = null;\n\t\tswitch (t.getSerializationType()) {\n\t\t\tcase Transition.RULE:\n\t\t\t\tRuleTransition ruleTransition = (RuleTransition)t;\n\t\t\t\tPredictionContext newContext =\n\t\t\t\t\tSingletonPredictionContext.create(config.context, ruleTransition.followState.stateNumber);\n\t\t\t\tc = new LexerATNConfig(config, t.target, newContext);\n\t\t\t\tbreak;\n\n\t\t\tcase Transition.PRECEDENCE:\n\t\t\t\tthrow new UnsupportedOperationException(\"Precedence predicates are not supported in lexers.\");\n\n\t\t\tcase Transition.PREDICATE:\n\t\t\t\t/*  Track traversing semantic predicates. If we traverse,\n\t\t\t\t we cannot add a DFA state for this \"reach\" computation\n\t\t\t\t because the DFA would not test the predicate again in the\n\t\t\t\t future. Rather than creating collections of semantic predicates\n\t\t\t\t like v3 and testing them on prediction, v4 will test them on the\n\t\t\t\t fly all the time using the ATN not the DFA. This is slower but\n\t\t\t\t semantically it's not used that often. One of the key elements to\n\t\t\t\t this predicate mechanism is not adding DFA states that see\n\t\t\t\t predicates immediately afterwards in the ATN. For example,\n\n\t\t\t\t a : ID {p1}? | ID {p2}? ;\n\n\t\t\t\t should create the start state for rule 'a' (to save start state\n\t\t\t\t competition), but should not create target of ID state. The\n\t\t\t\t collection of ATN states the following ID references includes\n\t\t\t\t states reached by traversing predicates. Since this is when we","sourceCodeStart":463,"sourceCodeEnd":499,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Java/src/org/antlr/v4/runtime/atn/LexerATNSimulator.java#L463-L499","documentation":"LexerATNSimulator.closure hits a PrecedencePredicateTransition while computing lexer DFA reach. Precedence predicates exist only for parser rules (left-recursive rule precedence, grammar option 'options { contextSuperClass... }' style precedence), and the lexer engine cannot evaluate them. Their presence in a lexer ATN means the grammar/tool produced something the lexer runtime cannot run.","triggerScenarios":"A lexer grammar whose generated ATN contains PrecedencePredicate transitions: using precedence operators/semantic predicates of the precedence form ({p}? <= ... is parser-only) in lexer rules, or feeding a parser ATN/serialized data to a lexer interpreter (e.g. building a LexerInterpreter over .interp data generated for a parser). Mismatched tool/runtime versions can also deserialize an ATN with unexpected transition types.","commonSituations":"Using the Grammar interpreter API (Grammar, LexerInterpreter) with an .interp file produced from a grammar that has left recursion in lexer mode or is actually a combined grammar misinterpreted as a lexer. Downgrading the runtime below the tool version so new ATN features appear in old runtime code paths.","solutions":["Remove precedence constructs (left-recursive-style precedence, precedence semantic predicates) from lexer rules; lexers only support plain semantic predicates {...}?","Verify you are not constructing a LexerInterpreter from parser .interp data; build a ParserInterpreter for parser grammars","Align the ANTLR tool and runtime versions, then regenerate all lexers/parsers"],"exampleFix":"// before\n// lexer grammar with precedence-style predicate (invalid in lexer)\nTOKEN: {precedence > 0}? <= 'a' 'b'+ ;\n// after\n// use a plain semantic predicate in the lexer rule\nTOKEN: {this.inParens()}? 'b'+ ;","handlingStrategy":"validation","validationCode":"// Before building a LexerInterpreter, confirm the serialized ATN has no PRECEDENCE transitions\nint[] data = /* serialized ints */;\nboolean hasPrecedence = new ATNDeserializer().deserialize(data) // inspect\n        .states.stream().filter(Objects::nonNull)\n        .flatMap(s -> s.transitions().stream())\n        .anyMatch(t -> t.getSerializationType() == Transition.PRECEDENCE);\nif (hasPrecedence) throw new IllegalArgumentException(\"grammar not usable as lexer\");","typeGuard":"static boolean isLexerSafe(ATN atn) {\n    for (ATNState s : atn.states) {\n        if (s == null) continue;\n        for (int i = 0; i < s.getNumberOfTransitions(); i++) {\n            if (s.transition(i) instanceof PrecedencePredicateTransition) return false;\n        }\n    }\n    return true;\n}","tryCatchPattern":null,"preventionTips":["Keep precedence constructs out of lexer rules","Use LexerInterpreter only with lexer .interp data","Match tool and runtime versions before shipping serialized ATNs"],"tags":["antlr","java","lexer","precedence","predicate","unsupported-operation"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}