antlr/antlr4 · error · IllegalStateException

A lexer interpreter can only be created for a lexer or combi

Error message

A lexer interpreter can only be created for a lexer or combined grammar.

What it means

Grammar.createLexerInterpreter(CharStream) builds a LexerInterpreter for a grammar, but a pure parser grammar (type PARSER, with an imported/external token file) has no lexer ATN to interpret, so it throws IllegalStateException. Combined grammars delegate to their implicit lexer.

Source

Thrown at tool/src/org/antlr/v4/tool/Grammar.java:1319

				stateToGrammarRegionMap.put(n.atnState.stateNumber, tokenRegion);
			}
		}
		return stateToGrammarRegionMap;
	}

	/** Given an ATN state number, return the token index range within the grammar from which that ATN state was derived. */
	public Interval getStateToGrammarRegion(int atnStateNumber) {
		if ( stateToGrammarRegionMap==null ) {
			stateToGrammarRegionMap = getStateToGrammarRegionMap(ast, null); // map all nodes with non-null atn state ptr
		}
		if ( stateToGrammarRegionMap==null ) return Interval.INVALID;

		return stateToGrammarRegionMap.get(atnStateNumber);
	}

	public LexerInterpreter createLexerInterpreter(CharStream input) {
		if (this.isParser()) {
			throw new IllegalStateException("A lexer interpreter can only be created for a lexer or combined grammar.");
		}

		if (this.isCombined()) {
			return implicitLexer.createLexerInterpreter(input);
		}

		List<String> allChannels = new ArrayList<String>();
		allChannels.add("DEFAULT_TOKEN_CHANNEL");
		allChannels.add("HIDDEN");
		allChannels.addAll(channelValueToNameList);

		// must run ATN through serializer to set some state flags
		IntegerList serialized = ATNSerializer.getSerialized(atn);
		ATN deserializedATN = new ATNDeserializer().deserialize(serialized.toArray());
		return new LexerInterpreter(
				fileName,
				getVocabulary(),
				Arrays.asList(getRuleNames()),

View on GitHub (pinned to 7d5770395b)

Solutions

1. Branch on grammar type: only call createLexerInterpreter when g.isLexer() or g.isCombined()
2. For a parser grammar, load its companion lexer grammar instead and interpret that
3. Consider the GrammarSpec API: check g.getType() first

Example fix

// before
LexerInterpreter lex = g.createLexerInterpreter(charStream);

// after
LexerInterpreter lex;
if (g.isParser() && !g.isCombined()) {
    g = loadLexerGrammarFor(g); // separate lexer .g4
}
lex = g.createLexerInterpreter(charStream);

boolean canLex = g.isLexer() || g.isCombined();

static boolean supportsLexerInterpreter(Grammar g) { return !g.isParser() || g.isCombined(); }

try { g.createLexerInterpreter(input); } catch (IllegalStateException e) { /* load the companion lexer grammar instead */ }

When it happens

Trigger: Calling g.createLexerInterpreter(input) where g.getType() == GrammarType.PARSER (grammar created from a parser-only .g4 with a TOKENS file or separate lexer grammar).

Common situations: Loading grammars via GrammarTool/ANTLRStringStorage at runtime and assuming every grammar can drive an interpreter; testing pipelines that uniformly call both createLexerInterpreter and createParserInterpreter on whatever grammar was loaded.

Related errors

• A parser interpreter can only be created for a parser or com
• Unexpected data entry
• can't create parser to match incoming {parserClass}
• replace op boundaries of ${rop} overlap with previous ${prev
• insert op ${iop} within boundaries of previous ${rop}

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/71765ff4f8700273. Report an issue: GitHub.