sgl-project/sglang · error · NotImplementedError

Host cache is not supported yet

Error message

Host cache is not supported yet

What it means

RadixCacheCpp.__init__ raises NotImplementedError('Host cache is not supported yet') when hierarchical cache (hicache) is configured with a cache controller. The experimental C++ tree only supports the pure-GPU path (it early-returns when hicache is unused); host/offloading integration is unimplemented.

Source

Thrown at python/sglang/srt/mem_cache/radix_cache_cpp.py:88

        self.page_size = params.page_size
        self.kv_cache = self.token_to_kv_pool_allocator.get_kvcache()

        self.tp_group = params.tp_cache_group

        if params.enable_metrics:
            self.init_metrics_collector()

        if not get_memory().enable_hierarchical_cache:
            self.tree = RadixTreeCpp(
                disabled=self.disable,
                page_size=self.page_size,
                host_size=None,  # no host cache, this should be removed in the future
                write_through_threshold=self.write_through_threshold,
            )
            self.cache_controller = None
            return  # early return if hicache is not used

        raise NotImplementedError("Host cache is not supported yet")

    def _merge_tensor(self, l: List[torch.Tensor]) -> torch.Tensor:
        """
        Merge a list of tensors into a single tensor.
        Args:
            l (List[torch.Tensor]): List of tensors to merge.
        Returns:
            torch.Tensor: Merged tensor.
        """
        if len(l) == 0:
            return torch.empty(0, dtype=torch.int64, device=self.device)
        elif len(l) == 1:
            return l[0]
        else:
            return torch.cat(l)

    def reset(self):
        if self.cache_controller is not None:

View on GitHub (pinned to 0132848349)

Solutions

1. Disable hierarchical cache when using the C++ radix tree
2. Or keep hicache and use the default Python RadixCache/HiRadixCache backend
3. Track upstream until host cache support lands in the C++ tree

Example fix

# before
--enable-hierarchical-cache --radix-cache-backend cpp
# after
--radix-cache-backend cpp  # no hicache

if server_args.enable_hierarchical_cache and radix_backend_is_cpp(server_args):
    raise ValueError("hicache + C++ radix tree unsupported")

When it happens

Trigger: Initializing RadixCacheCpp with enable_hierarchical_cache=True (or any config that yields a non-None cache controller / host pool), i.e. combining --enable-hierarchical-cache with the C++ radix backend.

Common situations: Trying to get C++ tree speed plus host offload for long-context workloads; enabling hicache globally in server args while testing the experimental C++ backend.

Related errors

• /v1/models ${response.status}
• Unsupported msgpack byte ${b}
• This browser does not support gzip stream decoding
• Ascend PD transfer does not support HiSparse destination dev
• get_split_heads_page_buffer_meta requires layout='page_head'

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/498ae6d443cd44ae. Report an issue: GitHub.