sgl-project/sglang · error · ValueError

RadixKey operations require matching cache_salt, but got {se

Error message

RadixKey operations require matching cache_salt, but got {self.cache_salt=} != {other.cache_salt=}

What it means

RadixKey binary operations require identical cache_salt on both keys. cache_salt namespaces the prefix cache (e.g. per-tenant or per-session salt) so entries from different salts never mix; _check_compatible raises ValueError when they differ.

Source

Thrown at python/sglang/srt/mem_cache/radix_cache.py:176

        is_eagle: bool,
        value: Optional[torch.Tensor] = None,
    ) -> Tuple[RadixKey, Optional[torch.Tensor]]:
        # O(1): flip the bigram flag instead of materializing a tuple list.
        # value is paired with raw tokens and gets truncated to the bigram count.
        if is_eagle and not self.is_bigram:
            self.is_bigram = True
            if value is not None:
                value = value[: len(self)]
        return self, value

    def _check_compatible(self, other: RadixKey) -> None:
        if self.extra_key != other.extra_key:
            raise ValueError(
                f"RadixKey operations require matching extra_key, but got "
                f"{self.extra_key=} != {other.extra_key=}"
            )
        if self.cache_salt != other.cache_salt:
            raise ValueError(
                f"RadixKey operations require matching cache_salt, but got "
                f"{self.cache_salt=} != {other.cache_salt=}"
            )

    def match(self, other: RadixKey, page_size: int = 1) -> int:
        """Logical-unit prefix length shared with ``other``. Result is rounded down to ``page_size``."""
        self._check_compatible(other)
        t0, t1 = self.token_ids, other.token_ids
        assert type(t0) is type(t1), (type(t0), type(t1))
        n = min(len(t0), len(t1))

        # Exponential search for the first diverging token: gallop in doubling
        # windows (one C-level slice compare each), then binary-search the window
        # holding the divergence -- no per-token Python loop on long shared prefixes.
        matched_tokens = n
        lo = 0
        step = 1
        while lo < n:

View on GitHub (pinned to 0132848349)

Solutions

1. Thread the request's cache_salt through every RadixKey construction on both insert and lookup paths
2. Set a consistent global salt (or per-tenant salt) instead of mixing salted and unsalted keys
3. Assert key.cache_salt equality in tests to catch drift early

Example fix

# before
probe = RadixKey(token_ids, extra_key=extra)  # salt lost
# after
probe = RadixKey(token_ids, extra_key=extra, cache_salt=req.cache_salt)

if key_a.cache_salt != key_b.cache_salt:
    raise ValueError("cache_salt mismatch")

def keys_compatible(a: RadixKey, b: RadixKey) -> bool:
    return a.extra_key == b.extra_key and a.cache_salt == b.cache_salt

When it happens

Trigger: Calling key_a.match(key_b) (directly or via match_prefix) where one key was created with cache_salt='tenant-a' and the other with a different or None salt.

Common situations: Enabling cache_salt on some requests but constructing probe keys without the salt; changing the salt between insert and lookup; multi-tenant deployments where the salt is dropped in a helper function.

Related errors

• RadixKey slice step must be 1
• RadixKey operations require matching extra_key, but got {sel
• cache_salt is not supported by the experimental C++ radix tr
• RadixKey index out of range: {idx}
• --radix-cache-backend={name!r} is not registered. Registered

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/92d772f02c0f3c80. Report an issue: GitHub.