antlr/antlr4 · error · ArgumentOutOfRangeException

get(${i}) outside buffer: ${bufferStartIndex}..${bufferStart

Error message

get(${i}) outside buffer: ${bufferStartIndex}..${bufferStartIndex+n}

What it means

UnbufferedTokenStream stores tokens in a rolling buffer. Get(i) performs random access only inside that current buffer; an absolute token index before the buffer start or at/after buffer start plus count is outside the retained window and throws ArgumentOutOfRangeException.

Source

Thrown at runtime/CSharp/src/UnbufferedTokenStream.cs:122

            : this(tokenSource, 256)
        {
        }

        public UnbufferedTokenStream(ITokenSource tokenSource, int bufferSize)
        {
            this.TokenSource = tokenSource;
            this.tokens = new IToken[bufferSize];
            n = 0;
            Fill(1);
        }

        // prime the pump
        public virtual IToken Get(int i)
        {
            int bufferStartIndex = GetBufferStartIndex();
            if (i < bufferStartIndex || i >= bufferStartIndex + n)
            {
                throw new ArgumentOutOfRangeException("get(" + i + ") outside buffer: " + bufferStartIndex + ".." + (bufferStartIndex + n));
            }
            return tokens[i - bufferStartIndex];
        }

        public virtual IToken LT(int i)
        {
            if (i == -1)
            {
                return lastToken;
            }
            Sync(i);
            int index = p + i - 1;
            if (index < 0)
            {
                throw new ArgumentOutOfRangeException("LT(" + i + ") gives negative index");
            }
            if (index >= n)
            {

View on GitHub (pinned to 7d5770395b)

Solutions

1. Use relative access such as LT(k) when streaming.
2. Call Get only while the token remains in the current buffer and only after sufficient fill.
3. Use BufferedTokenStream when arbitrary token indexing is required.

Example fix

// before
IToken t = unbuffered.Get(0); // after buffer advanced

// after
var tokens = new BufferedTokenStream(lexer);
tokens.Fill();
IToken t = tokens.Get(0);

if (tokens is UnbufferedTokenStream)
    throw new InvalidOperationException("Random token access requires BufferedTokenStream.");

static bool SupportsRandomTokenAccess(ITokenStream stream) => stream is not UnbufferedTokenStream;

try { var t = tokens.Get(i); }
catch (ArgumentOutOfRangeException ex) when (ex.Message.StartsWith("get(")) { /* use LT(k) or BufferedTokenStream */ }

When it happens

Trigger: Calling Get(index) for already flushed tokens; requesting an absolute token index before the stream has synchronized/filled that far; or reusing an index after the buffer advanced.

Common situations: Token-inspection utilities written for BufferedTokenStream; rewrite/listener phases that index old tokens; debugging code that jumps to token numbers; or early EOF with indexes assumed to exist.

Related errors

• cannot consume EOF
• Unbuffered stream cannot know its size
• get({i}) outside buffer: {bufferStartIndex}..{bufferStartInd
• seek to index outside buffer: {index} not in {bufferStartInd
• Unbuffered stream cannot know its size

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/994dd3e139c3c805. Report an issue: GitHub.