antlr/antlr4 · error · IndexOutOfBoundsException

LT({i}) gives negative index

Error message

LT({i}) gives negative index

What it means

UnbufferedTokenStream.LT(i) computes index = p + i - 1, where p is the current buffer position. If i is negative enough that this index falls below zero, the requested lookbehind reaches before the start of the buffer (or before the stream start) and LT throws IndexOutOfBoundsException. Lookahead forward is synced automatically; lookbehind is limited to what the window retains.

Source

Thrown at runtime/Java/src/org/antlr/v4/runtime/UnbufferedTokenStream.java:97

	public Token get(int i) { // get absolute index
		int bufferStartIndex = getBufferStartIndex();
		if (i < bufferStartIndex || i >= bufferStartIndex + n) {
			throw new IndexOutOfBoundsException("get("+i+") outside buffer: "+
			                    bufferStartIndex+".."+(bufferStartIndex+n));
		}
		return tokens[i - bufferStartIndex];
	}

	@Override
	public Token LT(int i) {
		if ( i==-1 ) {
			return lastToken;
		}

		sync(i);
        int index = p + i - 1;
        if ( index < 0 ) {
			throw new IndexOutOfBoundsException("LT("+i+") gives negative index");
		}

		if ( index >= n ) {
			assert n > 0 && tokens[n-1].getType() == Token.EOF;
			return tokens[n-1];
		}

		return tokens[index];
	}

	@Override
	public int LA(int i) {
		return LT(i).getType();
	}

	@Override
	public TokenSource getTokenSource() {
		return tokenSource;

View on GitHub (pinned to 7d5770395b)

Solutions

1. Limit lookbehind: use LT(-1) (always valid via lastToken) and avoid deeper negative k with unbuffered streams
2. Hold a mark() at the point from which you may need to look back, keeping those tokens in the window
3. Switch to CommonTokenStream if the grammar or error strategy needs unbounded backward lookahead

Example fix

// before
Token prev = tokens.LT(-5); // may be before buffer start -> throws

// after
int m = tokens.mark(); // pin window before consuming
try {
    consumeSome(tokens);
    Token prev = tokens.LT(-5); // within marked window
} finally {
    tokens.release(m);
}

static Token lookBack(TokenStream tokens, int k) {
    return (k == 1) ? tokens.LT(-1) // always valid via lastToken
        : (k <= 0) ? tokens.LT(k)      // 0 and positive are safe/synced
        : nullIfTooFar(tokens, k);     // guard deeper lookbehind yourself
}

try { Token t = tokens.LT(i); }
catch (IndexOutOfBoundsException e) { /* lookbehind before buffer start: use LT(-1) or a buffered stream */ }

When it happens

Trigger: LT(-k) where k exceeds the tokens retained behind the current position (markers released, window advanced); LT with a very large negative i on a fresh stream; custom error handling that inspects preceding tokens beyond the window.

Common situations: Lexer/parser customization that walks back many tokens for context; using UnbufferedTokenStream where CommonTokenStream's unlimited lookbehind was previously available; LT(-1) before the first token is fine (returns lastToken), but deeper lookbehind can fail.

Related errors

• get({i}) outside buffer: {bufferStartIndex}..{bufferStartInd
• interval {interval} not in token buffer window: {bufferStart
• tokenSource cannot be null
• cannot consume EOF
• token index {} out of range 0..{}

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/b4b9da1430dd8808. Report an issue: GitHub.