{"record":{"id":"f5ba4911cbba558a","repo":"sgl-project/sglang","slug":"hisparse-device-kv-transfer-requires-sgl-kernel-kv","errorCode":null,"errorMessage":"HiSparse device KV transfer requires sgl_kernel.kvcacheio (CUDA/ROCm). It is not available on this backend.","messagePattern":"HiSparse device KV transfer requires sgl_kernel\\.kvcacheio \\(CUDA/ROCm\\)\\. It is not available on this backend\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/mem_cache/hisparse_memory_pool.py","lineNumber":22,"sourceCode":"from typing import Optional\n\nimport torch\n\nfrom sglang.srt.layers.radix_attention import RadixAttention\nfrom sglang.srt.mem_cache.memory_pool import DSATokenToKVPool\nfrom sglang.srt.utils import is_cuda, is_hip\n\nlogger = logging.getLogger(__name__)\n\n# sgl_kernel.kvcacheio is only available in CUDA/ROCm sgl-kernel builds (not XPU/MPS/NPU/CPU).\n_is_cuda = is_cuda()\n_is_hip = is_hip()\nif _is_cuda or _is_hip:\n    from sgl_kernel.kvcacheio import transfer_kv_all_layer_mla\nelse:\n\n    def transfer_kv_all_layer_mla(*args, **kwargs):\n        raise RuntimeError(\n            \"HiSparse device KV transfer requires sgl_kernel.kvcacheio (CUDA/ROCm). \"\n            \"It is not available on this backend.\"\n        )\n\n\nclass HiSparseDSATokenToKVPool(DSATokenToKVPool):\n    def __init__(\n        self,\n        size: int,\n        page_size: int,\n        kv_lora_rank: int,\n        dtype: torch.dtype,\n        qk_rope_head_dim: int,\n        layer_num: int,\n        device: str,\n        index_head_dim: int,\n        enable_memory_saver: bool,\n        kv_cache_dim: int,","sourceCodeStart":4,"sourceCodeEnd":40,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/hisparse_memory_pool.py#L4-L40","documentation":"hisparse_memory_pool.py imports transfer_kv_all_layer_mla from sgl_kernel.kvcacheio only when running on CUDA or HIP; on any other backend (CPU, XPU, etc.) a stub is defined that raises this RuntimeError whenever on-device MLA KV transfer is attempted (transfer_values_on_device, backup_from_device_all_layer). The fused kernel simply doesn't exist for non-CUDA/ROCm platforms.","triggerScenarios":"Running a HiSparse/DSA workload with device-side KV transfer (transfer_values_on_device or backup_from_device_all_layer paths) on hardware that is not CUDA or ROCm, or with an sgl_kernel build that lacks kvcacheio.","commonSituations":"Trying HiSparse + DSA/MLA on CPU or an accelerator backend; a CPU-only dev install of sgl_kernel; platform detection (is_cuda/is_hip) misidentifying the backend in custom builds.","solutions":["Run on a CUDA (NVIDIA) or ROCm (AMD) GPU backend where sgl_kernel.kvcacheio is available","Install/upgrade the matching sgl_kernel wheel built with kvcacheio support for your GPU platform","If you're on unsupported hardware, avoid the device-transfer code paths (disable the HiSparse feature that triggers transfer_values_on_device/backup_from_device_all_layer)"],"exampleFix":"# before: running on CPU\nlaunch_server(..., enable_hisparse=True)  # DSA on-device transfer -> RuntimeError\n\n# after: use a CUDA machine\n# CUDA_VISIBLE_DEVICES=0 launch_server(..., enable_hisparse=True)","handlingStrategy":"fallback","validationCode":"import sglang.srt.utils as srt_utils\nok = srt_utils.is_cuda() or srt_utils.is_hip()\ntry:\n    import sgl_kernel.kvcacheio  # noqa\nexcept ImportError:\n    ok = False\nif not ok:\n    raise SystemExit(\"HiSparse device KV transfer needs CUDA/ROCm with sgl_kernel.kvcacheio\")","typeGuard":"def device_kv_transfer_available() -> bool:\n    try:\n        import sgl_kernel.kvcacheio  # noqa\n        return True\n    except ImportError:\n        return False","tryCatchPattern":"try:\n    pool.transfer_values_on_device(...)\nexcept RuntimeError as e:\n    if \"sgl_kernel.kvcacheio\" in str(e):\n        logger.error(\"GPU backend required for device KV transfer; disabling this path\")\n        disable_device_transfer = True\n    else:\n        raise","preventionTips":["Gate HiSparse device-transfer features on is_cuda()/is_hip() plus an sgl_kernel.kvcacheio import probe","Install the GPU-specific sgl_kernel wheel matching your torch/CUDA version"],"tags":["sgl-kernel","cuda","rocm","platform-support","hisparse"],"backgroundTag":"kernel-unavailable-on-backend","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}