{"record":{"id":"4a83191f3ce8102a","repo":"antlr/antlr4","slug":"token-index-out-of-range-0","errorCode":null,"errorMessage":"token index {} out of range 0..{}","messagePattern":"token index (.+?) out of range 0\\.\\.(.+?)","errorType":"exception","errorClass":"IndexOutOfBoundsException","httpStatus":null,"severity":"error","filePath":"runtime/Java/src/org/antlr/v4/runtime/BufferedTokenStream.java","lineNumber":186,"sourceCode":"        for (int i = 0; i < n; i++) {\n            Token t = tokenSource.nextToken();\n            if ( t instanceof WritableToken ) {\n                ((WritableToken)t).setTokenIndex(tokens.size());\n            }\n            tokens.add(t);\n            if ( t.getType()==Token.EOF ) {\n\t\t\t\tfetchedEOF = true;\n\t\t\t\treturn i + 1;\n\t\t\t}\n        }\n\n\t\treturn n;\n    }\n\n    @Override\n    public Token get(int i) {\n        if ( i < 0 || i >= tokens.size() ) {\n            throw new IndexOutOfBoundsException(\"token index \"+i+\" out of range 0..\"+(tokens.size()-1));\n        }\n        return tokens.get(i);\n    }\n\n\t/** Get all tokens from start..stop inclusively */\n\tpublic List<Token> get(int start, int stop) {\n\t\tif ( start<0 || stop<0 ) return null;\n\t\tlazyInit();\n\t\tList<Token> subset = new ArrayList<Token>();\n\t\tif ( stop>=tokens.size() ) stop = tokens.size()-1;\n\t\tfor (int i = start; i <= stop; i++) {\n\t\t\tToken t = tokens.get(i);\n\t\t\tif ( t.getType()==Token.EOF ) break;\n\t\t\tsubset.add(t);\n\t\t}\n\t\treturn subset;\n\t}\n","sourceCodeStart":168,"sourceCodeEnd":204,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Java/src/org/antlr/v4/runtime/BufferedTokenStream.java#L168-L204","documentation":"BufferedTokenStream.get(int i) validates the index against the current token buffer and throws IndexOutOfBoundsException for any i < 0 or i >= tokens.size(). The buffer only contains tokens fetched so far (lazily), so an index that looks valid for the whole input may still be out of range if EOF has not been reached yet. Note tokens.size()-1 in the message: with EOF fetched, the last valid index equals size()-1.","triggerScenarios":"Calling get(i) with an index computed from another stream's size(); calling get(size()) expecting it to be valid; calling get() before the stream has been fully consumed/fetched, so tokens.size() is smaller than the final token count.","commonSituations":"Token-inspection utilities, syntax highlighters, and error reporters that index tokens by absolute position; mixing indices obtained from Token.getTokenIndex() of a different token stream; off-by-one loops like for (i = 0; i <= size(); i++).","solutions":["Bound the loop with i < tokens.size() (not <=)","Call tokens.fill() first if you need to index over the entire input, then use size() as the bound","Validate 0 <= i < tokens.size() before get(i) when the index comes from external data"],"exampleFix":"// before\nfor (int i = 0; i <= tokens.size(); i++) { Token t = tokens.get(i); } // IOOBE on last i\n\n// after\ntokens.fill();\nfor (int i = 0; i < tokens.size(); i++) { Token t = tokens.get(i); }","handlingStrategy":"validation","validationCode":"int i = ...;\nif (i < 0 || i >= tokens.size()) throw new IllegalArgumentException(\"bad token index \" + i);\nToken t = tokens.get(i);","typeGuard":null,"tryCatchPattern":"try { Token t = tokens.get(i); } catch (IndexOutOfBoundsException e) { /* log and skip index from external input */ }","preventionTips":["Bound loops with i < size(), never i <= size()","Call tokens.fill() before random-access indexing over the whole input","Only use token indices obtained from tokens of the same stream"],"tags":["antlr","token-stream","index-out-of-bounds","off-by-one"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}