sgl-project/sglang · error · NotImplementedError

evictable_size() is not implemented; use full_evictable_size

Error message

evictable_size() is not implemented; use full_evictable_size() and swa_evictable_size() instead

What it means

SWARadixCache maintains two separate LRU domains (full and SWA), so a single evictable_size() is ambiguous and intentionally not implemented. Callers must use full_evictable_size() and swa_evictable_size().

Source

Thrown at python/sglang/srt/mem_cache/swa_radix_cache.py:855

                    self.token_to_kv_pool_allocator.free_swa(node.value)
                    self.swa_lru_list.remove_node(node)
                    node.swa_tombstone = True
                else:
                    # Internal: standard protected -> evictable.
                    self.swa_evictable_size_ += len(node.value)
            node.swa_lock_ref -= 1

            if swa_uuid_for_lock and node.swa_uuid == swa_uuid_for_lock:
                break
            node = node.parent

    def sanity_check(self):
        self.full_lru_list.sanity_check(self)
        self.swa_lru_list.sanity_check(self)

    def evictable_size(self) -> Tuple[int, int]:
        # Note: use full_evictable_size() and swa_evictable_size() instead.
        raise NotImplementedError

    def full_evictable_size(self) -> int:
        return self.full_evictable_size_

    def swa_evictable_size(self) -> int:
        return self.swa_evictable_size_

    def protected_size(self) -> Tuple[int, int]:
        # Note: use full_protected_size() and swa_protected_size() instead.
        raise NotImplementedError

    def full_protected_size(self) -> int:
        # protected size refers to the size of the full cache that is locked
        return self.full_protected_size_

    def swa_protected_size(self) -> int:
        # protected size refers to the size of the swa cache that is locked
        return self.swa_protected_size_

View on GitHub (pinned to 0132848349)

Solutions

1. Replace calls with full_evictable_size() and/or swa_evictable_size()
2. When writing cache-agnostic code, getattr-guard for the split accessors

Example fix

# before
size = tree.evictable_size()
# after
full = tree.full_evictable_size()
swa = tree.swa_evictable_size()

assert not hasattr(type(tree), 'swa_evictable_size') or False  # SWA cache detected
if hasattr(tree, 'swa_evictable_size'):
    full, swa = tree.full_evictable_size(), tree.swa_evictable_size()

def is_swa_cache(tree) -> bool:
    return hasattr(tree, 'full_evictable_size') and hasattr(tree, 'swa_evictable_size')

When it happens

Trigger: Calling evictable_size() on a SWARadixCache — code written against the BasePrefixCache interface that assumes a single evictable size.

Common situations: Generic cache-monitoring/metrics code that works for non-SWA RadixCache and is reused with SWA enabled models.

Related errors

• protected_size() is not implemented; use full_protected_size
• {self.__class__.__name__}.apply should not be called.
• Use get_key_buffer instead.
• Host cache is not supported yet
• SWA Radix tree sanity check failed, ping @hanming-lu: {e}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fbdf165fa26200ce. Report an issue: GitHub.