{"record":{"id":"7c109b54f5012182","repo":"sgl-project/sglang","slug":"layer-sharded-mla-hicache-backup-with-page-first-l","errorCode":null,"errorMessage":"Layer-sharded MLA HiCache backup with page_first layout requires the JIT one-layer kernel.","messagePattern":"Layer-sharded MLA HiCache backup with page_first layout requires the JIT one-layer kernel\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/pool_host/mla.py","lineNumber":384,"sourceCode":"                else:\n                    transfer_kv_per_layer_mla(\n                        src=device_pool.kv_buffer[device_layer_id],\n                        dst=self.kv_buffer[host_layer_id],\n                        src_indices=device_indices,\n                        dst_indices=host_indices,\n                        item_size=self.token_stride_size,\n                    )\n            elif self.layout == \"page_first\":\n                if self.can_use_jit:\n                    jit_transfer_hicache_one_layer_mla(\n                        cache_dst=self.data_refs[host_layer_id],\n                        cache_src=device_pool.kv_buffer[device_layer_id],\n                        indices_dst=host_indices,\n                        indices_src=device_indices,\n                        element_dim=self.kv_cache_dim,\n                    )\n                else:\n                    raise ValueError(\n                        \"Layer-sharded MLA HiCache backup with page_first layout \"\n                        \"requires the JIT one-layer kernel.\"\n                    )\n            else:\n                raise ValueError(\n                    f\"Layer-sharded HiCache backup does not support layout: {self.layout}\"\n                )\n        elif io_backend == \"direct\":\n            if self.layout == \"layer_first\":\n                transfer_kv_direct(\n                    src_layers=[device_pool.kv_buffer[device_layer_id]],\n                    dst_layers=[self.kv_buffer[host_layer_id]],\n                    src_indices=device_indices,\n                    dst_indices=host_indices,\n                    page_size=self.page_size,\n                )\n            else:\n                raise ValueError(","sourceCodeStart":366,"sourceCodeEnd":402,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/pool_host/mla.py#L366-L402","documentation":"During layer-sharded (per-layer) MLA HiCache backup from device, the page_first layout requires a JIT-compiled one-layer transfer kernel (transfer_kv_one_layer or similar). That kernel is unavailable — typically because sgl-kernel / the JIT kernel was not built or could not be imported — so the backup cannot proceed.","triggerScenarios":"Calling backup_from_device_all_layer / _backup_from_device_per_layer with a layer-sharded device pool and layout='page_first' when the JIT one-layer kernel is not available in the environment.","commonSituations":"Running a source install without compiled sgl-kernel JIT kernels; CUDA/arch mismatch preventing JIT compilation; missing kernel after an upgrade; using a container without the built extensions.","solutions":["Install/rebuild sgl-kernel so the JIT one-layer transfer kernel is available (pip install sgl-kernel matching your torch/CUDA, or python -m sglang.gen_jit... / build the JIT kernels)","Or switch io_backend to 'direct' for the backup path, which does not need the JIT kernel","Or use a non-layer-sharded device pool / layer_first layout","Verify torch/CUDA compatibility of the installed kernels"],"exampleFix":"// before\n# JIT kernel missing -> backup_from_device_all_layer raises\n\n// after\npip install -U sgl-kernel  # rebuild so the JIT one-layer kernel exists\n# or avoid the kernel path:\npool.backup_from_device_all_layer(..., io_backend=\"direct\")","handlingStrategy":"fallback","validationCode":"from sglang.srt.mem_cache import transfer_utils  # or the kernel module\njit_ok = _jit_one_layer_kernel_available()\nif not jit_ok and layout == \"page_first\":\n    io_backend = \"direct\"","typeGuard":"def jit_one_layer_kernel_available() -> bool:\n    try:\n        from sglang.srt.mem_cache.transfer_utils import transfer_kv_one_layer  # noqa\n        return True\n    except Exception:\n        return False","tryCatchPattern":"try:\n    pool.backup_from_device_all_layer(...)\nexcept ValueError as e:\n    if \"requires the JIT one-layer kernel\" in str(e):\n        pool.backup_from_device_all_layer(..., io_backend=\"direct\")\n    else:\n        raise","preventionTips":["Verify sgl-kernel/JIT kernels import successfully at server startup","Keep a direct-IO fallback path for backup transfers","Pin matching torch/CUDA/sgl-kernel versions in deployment images"],"tags":["sglang","mla","hicache","jit-kernel","sgl-kernel","build"],"backgroundTag":"missing-compiled-kernel","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}