antlr/antlr4 · error · IllegalStateException

cannot consume EOF

Error message

cannot consume EOF

What it means

RuleTagToken represents a rule reference tag (e.g. <expr>) inside a compiled pattern; its constructor requires the rule name because the token's text, and its matching behavior against the ATN's bypass tokens, are derived from that name. A null or empty ruleName would make the token unmatchable and its getText meaningless, so IllegalArgumentException is thrown at construction.

Source

Thrown at runtime/Java/src/org/antlr/v4/runtime/ANTLRInputStream.java:130

   		}
   		finally {
   			r.close();
   		}
   	}

	/** Reset the stream so that it's in the same state it was
	 *  when the object was created *except* the data array is not
	 *  touched.
	 */
	public void reset() {
		p = 0;
	}

    @Override
    public void consume() {
		if (p >= n) {
			assert LA(1) == IntStream.EOF;
			throw new IllegalStateException("cannot consume EOF");
		}

		//System.out.println("prev p="+p+", c="+(char)data[p]);
        if ( p < n ) {
            p++;
			//System.out.println("p moves to "+p+" (c='"+(char)data[p]+"')");
        }
    }

    @Override
    public int LA(int i) {
		if ( i==0 ) {
			return 0; // undefined
		}
		if ( i<0 ) {
			i++; // e.g., translate LA(-1) to use offset i=0; then data[p+0-1]
			if ( (p+i-1) < 0 ) {
				return IntStream.EOF; // invalid; no char before first char

View on GitHub (pinned to 7d5770395b)

Solutions

1. Pass the grammar's rule name exactly as declared (e.g. new RuleTagToken("expr", bypassType, null)).
2. Prefer letting ParseTreePatternMatcher.compile create rule tags for you.
3. Validate non-empty name at the boundary where tokens are reconstructed from external data.

Example fix

// before
RuleTagToken t = new RuleTagToken(nameFromJson, bypassType, label); // may be ""

// after
if (nameFromJson == null || nameFromJson.isEmpty()) throw new IllegalArgumentException("rule name required");
RuleTagToken t = new RuleTagToken(nameFromJson, bypassType, label);

if (ruleName == null || ruleName.isEmpty()) {
    throw new IllegalArgumentException("ruleName required for RuleTagToken");
}
return new RuleTagToken(ruleName, bypassTokenType, label);

boolean isValidRuleName(String n) { return n != null && !n.isEmpty(); }

When it happens

Trigger: Constructing RuleTagToken(null, type, label) or RuleTagToken("", ...) directly; indirectly hard to reach because ParseTreePatternMatcher validates rule names against the grammar before creating RuleTagToken.

Common situations: Extending the pattern-matching API or writing tests that hand-assemble token streams containing rule tags; serialization code that rebuilds tokens from strings and passes an empty name when the source field was absent.

Related errors

• index cannot be negative
• missing interface implementation
• Fatal error occured while evaluating the names of the gramma
• ANTLR 4 caught {} build errors.
• tag cannot be null or empty

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/3cfc23e52cef26c8. Report an issue: GitHub.