antlr/antlr4 · error · IndexOutOfBoundsException

token index {} out of range 0..{}

Error message

token index {} out of range 0..{}

What it means

BufferedTokenStream.get(int i) validates the index against the current token buffer and throws IndexOutOfBoundsException for any i < 0 or i >= tokens.size(). The buffer only contains tokens fetched so far (lazily), so an index that looks valid for the whole input may still be out of range if EOF has not been reached yet. Note tokens.size()-1 in the message: with EOF fetched, the last valid index equals size()-1.

Source

Thrown at runtime/Java/src/org/antlr/v4/runtime/BufferedTokenStream.java:186

        for (int i = 0; i < n; i++) {
            Token t = tokenSource.nextToken();
            if ( t instanceof WritableToken ) {
                ((WritableToken)t).setTokenIndex(tokens.size());
            }
            tokens.add(t);
            if ( t.getType()==Token.EOF ) {
				fetchedEOF = true;
				return i + 1;
			}
        }

		return n;
    }

    @Override
    public Token get(int i) {
        if ( i < 0 || i >= tokens.size() ) {
            throw new IndexOutOfBoundsException("token index "+i+" out of range 0.."+(tokens.size()-1));
        }
        return tokens.get(i);
    }

	/** Get all tokens from start..stop inclusively */
	public List<Token> get(int start, int stop) {
		if ( start<0 || stop<0 ) return null;
		lazyInit();
		List<Token> subset = new ArrayList<Token>();
		if ( stop>=tokens.size() ) stop = tokens.size()-1;
		for (int i = start; i <= stop; i++) {
			Token t = tokens.get(i);
			if ( t.getType()==Token.EOF ) break;
			subset.add(t);
		}
		return subset;
	}

View on GitHub (pinned to 7d5770395b)

Solutions

1. Bound the loop with i < tokens.size() (not <=)
2. Call tokens.fill() first if you need to index over the entire input, then use size() as the bound
3. Validate 0 <= i < tokens.size() before get(i) when the index comes from external data

Example fix

// before
for (int i = 0; i <= tokens.size(); i++) { Token t = tokens.get(i); } // IOOBE on last i

// after
tokens.fill();
for (int i = 0; i < tokens.size(); i++) { Token t = tokens.get(i); }

int i = ...;
if (i < 0 || i >= tokens.size()) throw new IllegalArgumentException("bad token index " + i);
Token t = tokens.get(i);

try { Token t = tokens.get(i); } catch (IndexOutOfBoundsException e) { /* log and skip index from external input */ }

When it happens

Trigger: Calling get(i) with an index computed from another stream's size(); calling get(size()) expecting it to be valid; calling get() before the stream has been fully consumed/fetched, so tokens.size() is smaller than the final token count.

Common situations: Token-inspection utilities, syntax highlighters, and error reporters that index tokens by absolute position; mixing indices obtained from Token.getTokenIndex() of a different token stream; off-by-one loops like for (i = 0; i <= size(); i++).

Related errors

• cannot consume EOF
• start {} or stop {} not in 0..{}
• {} not in 0..{}
• get({i}) outside buffer: {bufferStartIndex}..{bufferStartInd
• tokenSource cannot be null

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/4a83191f3ce8102a. Report an issue: GitHub.