antlr/antlr4 · error · ArgumentOutOfRangeException

token index {i} out of range 0..{tokens.Count - 1}

Error message

token index {i} out of range 0..{tokens.Count - 1}

What it means

BufferedTokenStream.Get(int i) does random access into the buffered token list and requires 0 <= i < tokens.Count. Unlike LA/LT, which fetch lazily, Get only sees tokens already buffered, so an index that looks valid for the source may still be out of range if the stream has not been fetched that far. The exception message reports the offending index and the valid 0..Count-1 range.

Source

Thrown at runtime/CSharp/src/BufferedTokenStream.cs:259

                if (t is IWritableToken)
                {
                    ((IWritableToken)t).TokenIndex = tokens.Count;
                }
                tokens.Add(t);
                if (t.Type == TokenConstants.EOF)
                {
                    fetchedEOF = true;
                    return i + 1;
                }
            }
            return n;
        }

        public virtual IToken Get(int i)
        {
            if (i < 0 || i >= tokens.Count)
            {
                throw new ArgumentOutOfRangeException("token index " + i + " out of range 0.." + (tokens.Count - 1));
            }
            return tokens[i];
        }

        /// <summary>Get all tokens from start..stop inclusively.</summary>
        /// <remarks>Get all tokens from start..stop inclusively.</remarks>
        public virtual IList<IToken> Get(int start, int stop)
        {
            if (start < 0 || stop < 0)
            {
                return null;
            }
            LazyInit();
            IList<IToken> subset = new List<IToken>();
            if (stop >= tokens.Count)
            {
                stop = tokens.Count - 1;
            }

View on GitHub (pinned to 7d5770395b)

Solutions

1. Clamp the index: if (i >= 0 && i < stream.Size) token = stream.Get(i)
2. Prefer LT(k)/LA(k) for lookahead and Get(TokenIndex) only with indices taken from real tokens
3. Call stream.Size (which forces fetching up to EOF) before doing index math if you need the full count

Example fix

// before
IToken tok = stream.Get(stopIndex + 1); // may be out of range

// after
IToken tok = stopIndex + 1 < stream.Size ? stream.Get(stopIndex + 1) : stream.Get(stream.Size - 1);

IToken t = i >= 0 && i < stream.Size ? stream.Get(i) : null;

try { tok = stream.Get(i); } catch (ArgumentOutOfRangeException) { tok = null; /* index beyond buffer */ }

When it happens

Trigger: Calling Get(i) with a negative index, an index >= tokens.Count (e.g. using LT(k).TokenIndex from a further position, or computing stopIndex+1 after a rule), or assuming Get fetches on demand like LA does.

Common situations: Error-reporting code that walks token context by absolute index without bounds checks; using Get(Size()) to look for a token past the end; mixing relative lookahead offsets (k) with absolute indices when calling Get.

Related errors

• start {start} or stop {stop} not in 0..{tokens.Count - 1}
• {tokenIndex} not in 0..{tokens.Count - 1}
• tokenSource cannot be null
• cannot consume EOF
• get(${i}) outside buffer: ${bufferStartIndex}..${bufferStart

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/cd36f378d02f2a09. Report an issue: GitHub.