{"record":{"id":"b4b9da1430dd8808","repo":"antlr/antlr4","slug":"lt-i-gives-negative-index","errorCode":null,"errorMessage":"LT({i}) gives negative index","messagePattern":"LT\\((.+?)\\) gives negative index","errorType":"exception","errorClass":"IndexOutOfBoundsException","httpStatus":null,"severity":"error","filePath":"runtime/Java/src/org/antlr/v4/runtime/UnbufferedTokenStream.java","lineNumber":97,"sourceCode":"\tpublic Token get(int i) { // get absolute index\n\t\tint bufferStartIndex = getBufferStartIndex();\n\t\tif (i < bufferStartIndex || i >= bufferStartIndex + n) {\n\t\t\tthrow new IndexOutOfBoundsException(\"get(\"+i+\") outside buffer: \"+\n\t\t\t                    bufferStartIndex+\"..\"+(bufferStartIndex+n));\n\t\t}\n\t\treturn tokens[i - bufferStartIndex];\n\t}\n\n\t@Override\n\tpublic Token LT(int i) {\n\t\tif ( i==-1 ) {\n\t\t\treturn lastToken;\n\t\t}\n\n\t\tsync(i);\n        int index = p + i - 1;\n        if ( index < 0 ) {\n\t\t\tthrow new IndexOutOfBoundsException(\"LT(\"+i+\") gives negative index\");\n\t\t}\n\n\t\tif ( index >= n ) {\n\t\t\tassert n > 0 && tokens[n-1].getType() == Token.EOF;\n\t\t\treturn tokens[n-1];\n\t\t}\n\n\t\treturn tokens[index];\n\t}\n\n\t@Override\n\tpublic int LA(int i) {\n\t\treturn LT(i).getType();\n\t}\n\n\t@Override\n\tpublic TokenSource getTokenSource() {\n\t\treturn tokenSource;","sourceCodeStart":79,"sourceCodeEnd":115,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Java/src/org/antlr/v4/runtime/UnbufferedTokenStream.java#L79-L115","documentation":"UnbufferedTokenStream.LT(i) computes index = p + i - 1, where p is the current buffer position. If i is negative enough that this index falls below zero, the requested lookbehind reaches before the start of the buffer (or before the stream start) and LT throws IndexOutOfBoundsException. Lookahead forward is synced automatically; lookbehind is limited to what the window retains.","triggerScenarios":"LT(-k) where k exceeds the tokens retained behind the current position (markers released, window advanced); LT with a very large negative i on a fresh stream; custom error handling that inspects preceding tokens beyond the window.","commonSituations":"Lexer/parser customization that walks back many tokens for context; using UnbufferedTokenStream where CommonTokenStream's unlimited lookbehind was previously available; LT(-1) before the first token is fine (returns lastToken), but deeper lookbehind can fail.","solutions":["Limit lookbehind: use LT(-1) (always valid via lastToken) and avoid deeper negative k with unbuffered streams","Hold a mark() at the point from which you may need to look back, keeping those tokens in the window","Switch to CommonTokenStream if the grammar or error strategy needs unbounded backward lookahead"],"exampleFix":"// before\nToken prev = tokens.LT(-5); // may be before buffer start -> throws\n\n// after\nint m = tokens.mark(); // pin window before consuming\ntry {\n    consumeSome(tokens);\n    Token prev = tokens.LT(-5); // within marked window\n} finally {\n    tokens.release(m);\n}","handlingStrategy":"validation","validationCode":"static Token lookBack(TokenStream tokens, int k) {\n    return (k == 1) ? tokens.LT(-1) // always valid via lastToken\n        : (k <= 0) ? tokens.LT(k)      // 0 and positive are safe/synced\n        : nullIfTooFar(tokens, k);     // guard deeper lookbehind yourself\n}","typeGuard":null,"tryCatchPattern":"try { Token t = tokens.LT(i); }\ncatch (IndexOutOfBoundsException e) { /* lookbehind before buffer start: use LT(-1) or a buffered stream */ }","preventionTips":["Prefer LT(-1) for one-token lookbehind; it never throws","Mark the stream before consuming if deeper lookbehind is needed","Use CommonTokenStream for grammars/tools requiring unbounded lookbehind"],"tags":["antlr","token-stream","lookbehind","lt","buffer"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}