antlr/antlr4 · error · UnsupportedOperationException

Parser can't discover a lexer to use

Error message

Parser can't discover a lexer to use

What it means

compileParseTreePattern(String, int) tries to auto-discover a Lexer by unwrapping the parser's token stream: getTokenStream().getTokenSource() must be a Lexer instance, because the pattern compiler lexes the pattern text with that same lexer grammar. If the token source is not a Lexer (e.g. a ListTokenSource or custom TokenSource), it throws UnsupportedOperationException. The overload taking an explicit Lexer never has this problem.

Source

Thrown at runtime/Java/src/org/antlr/v4/runtime/Parser.java:480

	 * The preferred method of getting a tree pattern. For example, here's a
	 * sample use:
	 *
	 * <pre>
	 * ParseTree t = parser.expr();
	 * ParseTreePattern p = parser.compileParseTreePattern("&lt;ID&gt;+0", MyParser.RULE_expr);
	 * ParseTreeMatch m = p.match(t);
	 * String id = m.get("ID");
	 * </pre>
	 */
	public ParseTreePattern compileParseTreePattern(String pattern, int patternRuleIndex) {
		if ( getTokenStream()!=null ) {
			TokenSource tokenSource = getTokenStream().getTokenSource();
			if ( tokenSource instanceof Lexer ) {
				Lexer lexer = (Lexer)tokenSource;
				return compileParseTreePattern(pattern, patternRuleIndex, lexer);
			}
		}
		throw new UnsupportedOperationException("Parser can't discover a lexer to use");
	}

	/**
	 * The same as {@link #compileParseTreePattern(String, int)} but specify a
	 * {@link Lexer} rather than trying to deduce it from this parser.
	 */
	public ParseTreePattern compileParseTreePattern(String pattern, int patternRuleIndex,
													Lexer lexer)
	{
		ParseTreePatternMatcher m = new ParseTreePatternMatcher(lexer, this);
		return m.compile(pattern, patternRuleIndex);
	}


	public ANTLRErrorStrategy getErrorHandler() {
		return _errHandler;
	}

View on GitHub (pinned to 7d5770395b)

Solutions

1. Call the three-argument overload with the original lexer: parser.compileParseTreePattern(pattern, ruleIndex, lexer)
2. Keep a reference to the Lexer you created and pass it explicitly instead of relying on discovery
3. If discovery is required, ensure the parser's token stream is built directly over that Lexer (CommonTokenStream(lexer))

Example fix

// before
parser.setTokenStream(new CommonTokenStream(listTokenSource));
parser.compileParseTreePattern("<ID>", R.rule); // throws

// after
parser.compileParseTreePattern("<ID>", R.rule, myLexer); // explicit lexer

TokenSource src = parser.getTokenStream() != null ? parser.getTokenStream().getTokenSource() : null;
if (src instanceof Lexer) {
  pattern = parser.compileParseTreePattern(pat, ruleIndex);
} else {
  pattern = parser.compileParseTreePattern(pat, ruleIndex, knownLexer);
}

Lexer discoverLexer(Parser p) {
  TokenSource s = p.getTokenStream() != null ? p.getTokenStream().getTokenSource() : null;
  return (s instanceof Lexer) ? (Lexer) s : null;
}

When it happens

Trigger: Calling the two-argument compileParseTreePattern() on a parser fed by new CommonTokenStream(new ListTokenSource(tokens, name)); using a custom TokenSource implementation; calling it when getTokenStream() is null (parser not wired to a stream yet).

Common situations: Token-rewriting pipelines that reparse filtered tokens; test harnesses feeding synthetic token lists; pattern-matching utilities that accept any parser regardless of how its input was produced.

Related errors

• The current parser does not support an ATN with bypass alter
• Precedence predicates are not supported in lexers.
• should only be one op per index
• nextToken requires a non-null input stream.
• tokens cannot be null

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/6c42e4b7d5143d7e. Report an issue: GitHub.