{"record":{"id":"7bc4a2001db3d23b","repo":"antlr/antlr4","slug":"precedence-predicates-are-not-supported-in-lexers-7bc4a2","errorCode":null,"errorMessage":"Precedence predicates are not supported in lexers.","messagePattern":"Precedence predicates are not supported in lexers\\.","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"runtime/Python3/src/antlr4/atn/LexerATNSimulator.py","lineNumber":372,"sourceCode":"                configs.add(config)\n\n        for t in config.state.transitions:\n            c = self.getEpsilonTarget(input, config, t, configs, speculative, treatEofAsEpsilon)\n            if c is not None:\n                currentAltReachedAcceptState = self.closure(input, c, configs, currentAltReachedAcceptState, speculative, treatEofAsEpsilon)\n\n        return currentAltReachedAcceptState\n\n    # side-effect: can alter configs.hasSemanticContext\n    def getEpsilonTarget(self, input:InputStream, config:LexerATNConfig, t:Transition, configs:ATNConfigSet,\n                                           speculative:bool, treatEofAsEpsilon:bool):\n        c = None\n        if t.serializationType==Transition.RULE:\n                newContext = SingletonPredictionContext.create(config.context, t.followState.stateNumber)\n                c = LexerATNConfig(state=t.target, config=config, context=newContext)\n\n        elif t.serializationType==Transition.PRECEDENCE:\n                raise UnsupportedOperationException(\"Precedence predicates are not supported in lexers.\")\n\n        elif t.serializationType==Transition.PREDICATE:\n                #  Track traversing semantic predicates. If we traverse,\n                # we cannot add a DFA state for this \"reach\" computation\n                # because the DFA would not test the predicate again in the\n                # future. Rather than creating collections of semantic predicates\n                # like v3 and testing them on prediction, v4 will test them on the\n                # fly all the time using the ATN not the DFA. This is slower but\n                # semantically it's not used that often. One of the key elements to\n                # this predicate mechanism is not adding DFA states that see\n                # predicates immediately afterwards in the ATN. For example,\n\n                # a : ID {p1}? | ID {p2}? ;\n\n                # should create the start state for rule 'a' (to save start state\n                # competition), but should not create target of ID state. The\n                # collection of ATN states the following ID references includes\n                # states reached by traversing predicates. Since this is when we","sourceCodeStart":354,"sourceCodeEnd":390,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Python3/src/antlr4/atn/LexerATNSimulator.py#L354-L390","documentation":"LexerATNSimulator.getEpsilonTarget() walks lexer ATN transitions and explicitly rejects transitions of type PRECEDENCE: '{' ... '}' precedence predicates are a parser (left-recursion) feature and have no defined semantics during lexing, so encountering one raises UnsupportedOperationException. It means the lexer ATN itself contains a precedence predicate transition, which stock grammars never produce.","triggerScenarios":"A lexer ATN containing precedence-predicate transitions — only reachable with malformed/hand-crafted serialized ATN data or a serious tool/runtime format mismatch; user code introspecting the lexer ATN and crossing such a transition.","commonSituations":"Attempting to use left-recursive precedence constructs in lexer rules; experimenting with hand-built ATNs; version-skew artifacts where a parser ATN is fed to the lexer simulator.","solutions":["Remove precedence predicates / left-recursion style constructs from lexer rules — use semantic predicates ({self.input.LA(1)...}) in the lexer instead.","Regenerate artifacts with matching tool/runtime versions to rule out a misdeserialized ATN.","Ensure you are not running LexerATNSimulator against a parser's serialized ATN."],"exampleFix":"// before (lexer rule using precedence-style construct -> unsupported)\n// expr: {prec}? ...  // parser-only feature\n\n// after (lexer-level lookahead via semantic predicate)\nFRAG: {self._input.LA(1) == ord('$')}? '$' ;","handlingStrategy":"validation","validationCode":"# keep precedence constructs out of lexer rules; use semantic predicates for lexer context\n# grep your grammar: lexer rules must not contain '{n,prec}?' style precedence predicates\nimport re\nbad = re.findall(r'^[A-Z_]+\\s*:[^;]*\\{\\d+[>,]', grammar_text, re.M)\nassert not bad, 'precedence predicate in lexer rule(s): %s' % bad","typeGuard":null,"tryCatchPattern":"from antlr4.error.Errors import LexerNoViableAltException  # unrelated; this error is UnsupportedOperationException\ntry:\n    lexer.nextToken()\nexcept UnsupportedOperationException as e:\n    if 'Precedence predicates' in str(e):\n        raise RuntimeError('grammar bug: precedence predicate used in a lexer rule')\n    raise","preventionTips":["Use precedence predicates only in parser rules","For lexer context sensitivity use semantic predicates over _input.LA(...)","Never feed a parser ATN into LexerATNSimulator"],"tags":["antlr4","python","lexer","precedence-predicate","unsupported-operation"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}