antlr/antlr4 · error · ArgumentOutOfRangeException

{tokenIndex} not in 0..{tokens.Count - 1}

Error message

{tokenIndex} not in 0..{tokens.Count - 1}

What it means

GetHiddenTokensToRight(tokenIndex, channel) returns the hidden-channel tokens following tokenIndex on the default channel; tokenIndex itself must address a real token in the buffered list (0 <= tokenIndex < tokens.Count). Because the method needs the surrounding on-channel context, an out-of-range anchor index is rejected immediately after LazyInit().

Source

Thrown at runtime/CSharp/src/BufferedTokenStream.cs:524

        }

        /// <summary>
        /// Collect all tokens on specified channel to the right of
        /// the current token up until we see a token on
        /// <see cref="Lexer.DefaultTokenChannel"/>
        /// or
        /// EOF. If
        /// <paramref name="channel"/>
        /// is
        /// <c>-1</c>
        /// , find any non default channel token.
        /// </summary>
        public virtual IList<IToken> GetHiddenTokensToRight(int tokenIndex, int channel)
        {
            LazyInit();
            if (tokenIndex < 0 || tokenIndex >= tokens.Count)
            {
                throw new ArgumentOutOfRangeException(tokenIndex + " not in 0.." + (tokens.Count - 1));
            }
            int nextOnChannel = NextTokenOnChannel(tokenIndex + 1, Lexer.DefaultTokenChannel);
            int to;
            int from = tokenIndex + 1;
            // if none onchannel to right, nextOnChannel=-1 so set to = last token
            if (nextOnChannel == -1)
            {
                to = Size - 1;
            }
            else
            {
                to = nextOnChannel;
            }
            return FilterForChannel(from, to, channel);
        }

        /// <summary>
        /// Collect all hidden tokens (any off-default channel) to the right of

View on GitHub (pinned to 7d5770395b)

Solutions

1. Validate 0 <= tokenIndex < stream.Size before the call
2. When probing neighbours, special-case the last token instead of calling with tokenIndex + 1
3. Capture indices from tokens of the same stream immediately before use, not across resets

Example fix

// before
var hidden = stream.GetHiddenTokensToRight(i + 1, Lexer.Hidden); // i+1 may equal Size

// after
var hidden = i + 1 < stream.Size ? stream.GetHiddenTokensToRight(i + 1, Lexer.Hidden) : null;

IList<IToken> hidden = tokenIndex >= 0 && tokenIndex < stream.Size
    ? stream.GetHiddenTokensToRight(tokenIndex, channel)
    : null;

When it happens

Trigger: Calling GetHiddenTokensToRight with a negative index, an index >= tokens.Count (e.g. tokenIndex+1 computed by the caller, or an index from a different stream), or passing a raw character offset instead of a token index.

Common situations: Whitespace/comment handling code that iterates tokens and probes neighbours past the last token; mixing editor character offsets with token indices; reusing indices captured before a stream reset.

Related errors

• token index {i} out of range 0..{tokens.Count - 1}
• start {start} or stop {stop} not in 0..{tokens.Count - 1}
• ${tokenIndex} not in 0..${len(tokens)-1}
• {} not in 0..{}
• tokenSource cannot be null

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/80c0cd6cfadc15c3. Report an issue: GitHub.