sgl-project/sglang · error · NotImplementedError

walk_radix_cache_for_canary does not support {cache_type.__n

Error message

walk_radix_cache_for_canary does not support {cache_type.__name__}

What it means

walk_radix_cache_for_canary dispatches on the exact runtime type of the radix cache: UnifiedRadixCache uses tree_core.walk_for_kv_canary, RadixCache/SWARadixCache use the manual walker. Any other cache class raises NotImplementedError because the walker does not know its node/lock layout.

Source

Thrown at python/sglang/srt/kv_canary/radix_cache_walker.py:38

    *,
    radix_cache: BasePrefixCache,
    unlocked_only: bool = False,
    swa_resident_only: bool = False,
) -> RadixCacheWalkResult:
    """Walk the radix tree and emit flat (slot_indices, positions, prev_slot_indices) tensors.

    With both flags False (default), emits every slot held by the radix cache (including slots
    also referenced by a currently-running req — that overlap is harmless redundancy with the
    per-forward HEAD/TAIL path). ``unlocked_only=True`` skips nodes still locked by a running
    req. ``swa_resident_only=True`` skips SWA-tombstoned nodes (slots evicted from the SWA
    window)."""
    cache_type = type(radix_cache)
    if cache_type is UnifiedRadixCache:
        return radix_cache.tree_core.walk_for_kv_canary(
            unlocked_only=unlocked_only, swa_resident_only=swa_resident_only
        )
    if cache_type is not RadixCache and cache_type is not SWARadixCache:
        raise NotImplementedError(
            f"walk_radix_cache_for_canary does not support {cache_type.__name__}"
        )

    slot_buf: list[int] = []
    position_buf: list[int] = []
    prev_slot_buf: list[int] = []

    _walk_radix_subtree(
        node=radix_cache.root_node,
        radix_cache=radix_cache,
        depth=0,
        parent_last_slot=-1,
        slot_buf=slot_buf,
        position_buf=position_buf,
        prev_slot_buf=prev_slot_buf,
        is_root=True,
        unlocked_only=unlocked_only,
        swa_resident_only=swa_resident_only,

View on GitHub (pinned to 0132848349)

Solutions

Pass a plain RadixCache, SWARadixCache, or UnifiedRadixCache instance
If you have a new cache type, extend walk_radix_cache_for_canary with a branch (or implement walk_for_kv_canary on its tree core) and upstream it
In tests, use a real RadixCache with dummy nodes rather than a mock

Example fix

// before
plan = walk_radix_cache_for_canary(custom_cache)
// after
plan = walk_radix_cache_for_canary(RadixCache(...))  # supported concrete type

Defensive patterns

Strategy: type-guard

Validate before calling

from sglang.srt.mem_cache.radix_cache import RadixCache, SWARadixCache
from sglang.srt.mem_cache.unified_radix_cache import UnifiedRadixCache
assert type(radix_cache) in (RadixCache, SWARadixCache, UnifiedRadixCache)

Type guard

from typing import Any

def is_walkable_cache(c: Any) -> bool:
    return type(c) in (RadixCache, SWARadixCache, UnifiedRadixCache)

Try / catch

try:
    walk_radix_cache_for_canary(cache)
except NotImplementedError:
    logger.warning('unsupported cache type; skipping canary sweep')

Prevention

Use concrete supported cache classes, not subclasses or mocks
When adding a new cache type, add its walker branch in the same PR

When it happens

Trigger: Calling walk_radix_cache_for_canary with a custom RadixCache subclass, a mocked cache, or a new cache type (e.g. HiRadixCache, ChunkCache-style variant) that is not exactly RadixCache, SWARadixCache, or UnifiedRadixCache.

Common situations: New radix cache subclass added to SGLang without a kv-canary walker; tests passing unittest.mock.MagicMock instead of a real cache instance; type(req_to_token_pool)-style subclasses breaking exact `is` checks.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fa9babde9d3a0356. Report an issue: GitHub.