{"record":{"id":"e5a32e984da1f2b8","repo":"antlr/antlr4","slug":"get-i-outside-buffer-bufferstartindex-buff","errorCode":null,"errorMessage":"get({i}) outside buffer: {bufferStartIndex}..{bufferStartIndex+n}","messagePattern":"get\\((.+?)\\) outside buffer: (.+?)\\.\\.(.+?)","errorType":"exception","errorClass":"IndexOutOfBoundsException","httpStatus":null,"severity":"error","filePath":"runtime/Java/src/org/antlr/v4/runtime/UnbufferedTokenStream.java","lineNumber":82,"sourceCode":"\t */\n\tprotected int currentTokenIndex = 0;\n\n\tpublic UnbufferedTokenStream(TokenSource tokenSource) {\n\t\tthis(tokenSource, 256);\n\t}\n\n\tpublic UnbufferedTokenStream(TokenSource tokenSource, int bufferSize) {\n\t\tthis.tokenSource = tokenSource;\n\t\ttokens = new Token[bufferSize];\n\t\tn = 0;\n\t\tfill(1); // prime the pump\n\t}\n\n\t@Override\n\tpublic Token get(int i) { // get absolute index\n\t\tint bufferStartIndex = getBufferStartIndex();\n\t\tif (i < bufferStartIndex || i >= bufferStartIndex + n) {\n\t\t\tthrow new IndexOutOfBoundsException(\"get(\"+i+\") outside buffer: \"+\n\t\t\t                    bufferStartIndex+\"..\"+(bufferStartIndex+n));\n\t\t}\n\t\treturn tokens[i - bufferStartIndex];\n\t}\n\n\t@Override\n\tpublic Token LT(int i) {\n\t\tif ( i==-1 ) {\n\t\t\treturn lastToken;\n\t\t}\n\n\t\tsync(i);\n        int index = p + i - 1;\n        if ( index < 0 ) {\n\t\t\tthrow new IndexOutOfBoundsException(\"LT(\"+i+\") gives negative index\");\n\t\t}\n\n\t\tif ( index >= n ) {","sourceCodeStart":64,"sourceCodeEnd":100,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Java/src/org/antlr/v4/runtime/UnbufferedTokenStream.java#L64-L100","documentation":"UnbufferedTokenStream.get(i) resolves absolute token indexes against the retained buffer window; if i is before bufferStartIndex (already discarded) or at/after bufferStartIndex + n (not yet buffered), it throws IndexOutOfBoundsException. The unbuffered token stream only supports lookaround within its window.","triggerScenarios":"get(i) for a token index far behind the current position after markers were released; get(i) ahead of the buffer without a preceding sync/fill; utilities that index tokens by absolute position (e.g., building a token list for an IDE).","commonSituations":"Error listeners or post-processing that fetch arbitrary historical tokens; porting code written against CommonTokenStream (which buffers everything) to UnbufferedTokenStream; token-annotating tools that assume random access.","solutions":["Use CommonTokenStream (buffered) when random access to all tokens is required","With the unbuffered stream, access tokens only via LT/LA relative to the current index, or hold marks to pin the window","Check the window bounds before calling get(): bufferStartIndex <= i < bufferStartIndex + n (expose bufferStartIndex as index() - p, or track via marks)"],"exampleFix":"// before\nTokenStream tokens = new UnbufferedTokenStream(lexer);\n// after parsing:\nToken first = tokens.get(0); // IndexOutOfBoundsException: already discarded\n\n// after\nTokenStream tokens = new CommonTokenStream(lexer); // buffers all tokens\nToken first = tokens.get(0);","handlingStrategy":"validation","validationCode":"int bufferStart = tokens.index() - /* p not exposed: track a floor via marks */ floorTokens;\nif (i >= bufferStart && i < bufferStart + bufferedCount) {\n    Token t = tokens.get(i);\n} else {\n    /* out of window: use cached tokens or a buffered stream */\n}","typeGuard":"static boolean supportsRandomAccess(TokenStream ts) {\n    return !(ts instanceof UnbufferedTokenStream);\n}","tryCatchPattern":"try { tokens.get(i); }\ncatch (IndexOutOfBoundsException e) { /* token discarded: re-parse with CommonTokenStream */ }","preventionTips":["Choose CommonTokenStream whenever tokens are indexed arbitrarily","Cache tokens you will need later instead of re-fetching from an unbuffered stream","Keep marks alive across any window of random access"],"tags":["antlr","token-stream","index-out-of-bounds","buffer","random-access"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}