{"record":{"id":"498ae6d443cd44ae","repo":"sgl-project/sglang","slug":"host-cache-is-not-supported-yet","errorCode":null,"errorMessage":"Host cache is not supported yet","messagePattern":"Host cache is not supported yet","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/radix_cache_cpp.py","lineNumber":88,"sourceCode":"        self.page_size = params.page_size\n        self.kv_cache = self.token_to_kv_pool_allocator.get_kvcache()\n\n        self.tp_group = params.tp_cache_group\n\n        if params.enable_metrics:\n            self.init_metrics_collector()\n\n        if not get_memory().enable_hierarchical_cache:\n            self.tree = RadixTreeCpp(\n                disabled=self.disable,\n                page_size=self.page_size,\n                host_size=None,  # no host cache, this should be removed in the future\n                write_through_threshold=self.write_through_threshold,\n            )\n            self.cache_controller = None\n            return  # early return if hicache is not used\n\n        raise NotImplementedError(\"Host cache is not supported yet\")\n\n    def _merge_tensor(self, l: List[torch.Tensor]) -> torch.Tensor:\n        \"\"\"\n        Merge a list of tensors into a single tensor.\n        Args:\n            l (List[torch.Tensor]): List of tensors to merge.\n        Returns:\n            torch.Tensor: Merged tensor.\n        \"\"\"\n        if len(l) == 0:\n            return torch.empty(0, dtype=torch.int64, device=self.device)\n        elif len(l) == 1:\n            return l[0]\n        else:\n            return torch.cat(l)\n\n    def reset(self):\n        if self.cache_controller is not None:","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/radix_cache_cpp.py#L70-L106","documentation":"RadixCacheCpp.__init__ raises NotImplementedError('Host cache is not supported yet') when hierarchical cache (hicache) is configured with a cache controller. The experimental C++ tree only supports the pure-GPU path (it early-returns when hicache is unused); host/offloading integration is unimplemented.","triggerScenarios":"Initializing RadixCacheCpp with enable_hierarchical_cache=True (or any config that yields a non-None cache controller / host pool), i.e. combining --enable-hierarchical-cache with the C++ radix backend.","commonSituations":"Trying to get C++ tree speed plus host offload for long-context workloads; enabling hicache globally in server args while testing the experimental C++ backend.","solutions":["Disable hierarchical cache when using the C++ radix tree","Or keep hicache and use the default Python RadixCache/HiRadixCache backend","Track upstream until host cache support lands in the C++ tree"],"exampleFix":"# before\n--enable-hierarchical-cache --radix-cache-backend cpp\n# after\n--radix-cache-backend cpp  # no hicache","handlingStrategy":"validation","validationCode":"if server_args.enable_hierarchical_cache and radix_backend_is_cpp(server_args):\n    raise ValueError(\"hicache + C++ radix tree unsupported\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep a compatibility matrix check in launch scripts","Prefer the Python backend when host offload is required"],"tags":["radix-cache","cpp-backend","hicache","not-implemented","host-offload"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}