What causes "RadixKey index out of range: {idx}" in sgl-project/sglang?

Check len(key) before indexing and clamp: idx = min(idx, len(key) - 1)

sgl-project/sglang · error · IndexError

RadixKey index out of range: {idx}

Error message

RadixKey index out of range: {idx}

What it means

RadixKey.__getitem__ raises IndexError when an integer index falls outside [0, len(key)) after negative-index normalization. RadixKey is the logical-unit key wrapper over token ids in SGLang's radix cache, so out-of-range slicing usually means caller code assumed a different key length (e.g. raw token count vs bigram count).

Source

Thrown at python/sglang/srt/mem_cache/radix_cache.py:124

    def __iter__(self) -> Iterator:
        t = self.token_ids
        n = self._raw_len()
        if self.is_bigram:
            for i in range(n - 1 if n > 0 else 0):
                yield (t[i], t[i + 1])
        elif n == len(t):
            yield from t
        else:
            for i in range(n):
                yield t[i]

    def __getitem__(self, idx: Union[int, slice]) -> RadixKey:
        # Normalize int -> 1-element slice so the rest handles one shape.
        if isinstance(idx, int):
            if idx < 0:
                idx += len(self)
            if idx < 0 or idx >= len(self):
                raise IndexError(f"RadixKey index out of range: {idx}")
            idx = slice(idx, idx + 1)
        start, stop, step = idx.indices(len(self))
        if step != 1:
            raise ValueError("RadixKey slice step must be 1")

        if self.is_bigram:
            # bigrams [start, stop) span raw tokens [start, stop + 1);
            # empty slice -> empty raw tokens (not a dangling boundary token).
            raw = self.token_ids[start : stop + 1] if stop > start else array("q")
            return RadixKey(
                raw,
                self.extra_key,
                is_bigram=True,
                cache_salt=self.cache_salt,
            )
        return RadixKey(
            self.token_ids[start:stop],
            self.extra_key,

View on GitHub (pinned to 0132848349)

Solutions

Check len(key) before indexing and clamp: idx = min(idx, len(key) - 1)
Use non-negative indices or ensure negative indices satisfy -len(key) <= idx < 0
For bigram keys, remember len is token count - 1 and derive indices from len(key), not from the raw token array

Example fix

# before
sub = key[len(token_ids) - 1]
# after
sub = key[min(len(token_ids) - 1, len(key) - 1)]

Defensive patterns

Strategy: validation

Validate before calling

n = len(key)
if not (-n <= idx < n):
    raise ValueError(f"index {idx} out of range for RadixKey of len {n}")

Prevention

Always derive indices from len(key), never from the raw token_ids length
Clamp indices before slicing
Write unit tests hitting both boundary indices 0 and len-1

When it happens

Trigger: Calling key[i] or key[i:j] on a RadixKey with i >= len(key), or a negative index whose magnitude exceeds len(key) (e.g. key[-5] on a length-2 key). Typically from prefix-matching code that computed an offset from token_ids length while the key is in bigram mode where len differs by 1.

Common situations: Off-by-one bugs when mixing raw token indices with logical (page/bigram) units; slicing with cached lengths after a key was truncated; porting code from the old array('q') token_ids interface to RadixKey.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a6e7913e38a88fef. Report an issue: GitHub.