{"record":{"id":"0c9d3cf8822aa4b6","repo":"antlr/antlr4","slug":"parser-can-t-discover-a-lexer-to-use-0c9d3c","errorCode":null,"errorMessage":"Parser can't discover a lexer to use","messagePattern":"Parser can't discover a lexer to use","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"runtime/Python3/src/antlr4/Parser.py","lineNumber":288,"sourceCode":"\n    # The preferred method of getting a tree pattern. For example, here's a\n    # sample use:\n    #\n    # <pre>\n    # ParseTree t = parser.expr();\n    # ParseTreePattern p = parser.compileParseTreePattern(\"&lt;ID&gt;+0\", MyParser.RULE_expr);\n    # ParseTreeMatch m = p.match(t);\n    # String id = m.get(\"ID\");\n    # </pre>\n    #\n    def compileParseTreePattern(self, pattern:str, patternRuleIndex:int, lexer:Lexer = None):\n        if lexer is None:\n            if self.getTokenStream() is not None:\n                tokenSource = self.getTokenStream().tokenSource\n                if isinstance( tokenSource, Lexer ):\n                    lexer = tokenSource\n        if lexer is None:\n            raise UnsupportedOperationException(\"Parser can't discover a lexer to use\")\n\n        m = ParseTreePatternMatcher(lexer, self)\n        return m.compile(pattern, patternRuleIndex)\n\n\n    def getInputStream(self):\n        return self.getTokenStream()\n\n    def setInputStream(self, input:InputStream):\n        self.setTokenStream(input)\n\n    def getTokenStream(self):\n        return self._input\n\n    # Set the token stream and reset the parser.#\n    def setTokenStream(self, input:TokenStream):\n        self._input = None\n        self.reset()","sourceCodeStart":270,"sourceCodeEnd":306,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Python3/src/antlr4/Parser.py#L270-L306","documentation":"Parser.compileParseTreePattern needs a Lexer to tokenize the pattern string (e.g., '<ID> + 0'). If the explicit lexer argument is None, it tries to discover one from the current token stream's tokenSource; when that source is not a Lexer instance (typical for ListTokenSource or a custom TokenSource), it raises UnsupportedOperationException('Parser can\\'t discover a lexer to use').","triggerScenarios":"Calling compileParseTreePattern(pattern, ruleIndex) without the third argument while the parser's input stream is fed by a ListTokenSource or other non-Lexer token source; constructing the parser from a pre-lexed token list for replay/testing; tokenSource wrapped in an adapter object.","commonSituations":"Test harnesses that replay recorded tokens; IDE/language-server code that parses from cached token lists; refactoring from a lexer-fed stream to a token-list stream while keeping tree-pattern code.","solutions":["Pass the lexer explicitly: parser.compileParseTreePattern(pattern, ruleIndex, MyLexer(None))","Keep the original lexer instance around when you switch the parser to a list-based token source and hand it to every pattern call","Alternatively build the parser on a CommonTokenStream over the real lexer so discovery succeeds"],"exampleFix":"# before\nparser = MyParser(CommonTokenStream(ListTokenSource(recorded_tokens)))\np = parser.compileParseTreePattern(\"<ID>\", MyParser.RULE_expr)  # no lexer to find\n\n# after\nlexer = MyLexer(None)\np = parser.compileParseTreePattern(\"<ID>\", MyParser.RULE_expr, lexer)","handlingStrategy":"validation","validationCode":"# Python: pass a lexer explicitly whenever the stream's source may not be one\nlexer = lexer or MyLexer(None)\npattern = parser.compileParseTreePattern(pattern_str, rule_index, lexer)","typeGuard":"# Python\nfrom antlr4.Lexer import Lexer\ndef token_source_is_lexer(parser) -> bool:\n    src = parser.getTokenStream().tokenSource if parser.getTokenStream() else None\n    return isinstance(src, Lexer)","tryCatchPattern":"try:\n    p = parser.compileParseTreePattern(pat, idx)\nexcept UnsupportedOperationException as ex:\n    if \"discover a lexer\" in str(ex):\n        p = parser.compileParseTreePattern(pat, idx, MyLexer(None))\n    else:\n        raise","preventionTips":["Store the lexer used for the parse and pass it to every pattern-compile call","When replaying tokens from a list, keep the original lexer instance alongside","Treat the implicit lexer discovery as a convenience only — make it explicit in production code"],"tags":["antlr","python","parser","tree-patterns","lexer-discovery"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}