{"record":{"id":"735331ab048a6aea","repo":"sgl-project/sglang","slug":"layer-sharded-direct-hicache-backup-only-supports","errorCode":null,"errorMessage":"Layer-sharded direct HiCache backup only supports layer_first layout, got {self.layout}","messagePattern":"Layer-sharded direct HiCache backup only supports layer_first layout, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/pool_host/mla.py","lineNumber":402,"sourceCode":"                    raise ValueError(\n                        \"Layer-sharded MLA HiCache backup with page_first layout \"\n                        \"requires the JIT one-layer kernel.\"\n                    )\n            else:\n                raise ValueError(\n                    f\"Layer-sharded HiCache backup does not support layout: {self.layout}\"\n                )\n        elif io_backend == \"direct\":\n            if self.layout == \"layer_first\":\n                transfer_kv_direct(\n                    src_layers=[device_pool.kv_buffer[device_layer_id]],\n                    dst_layers=[self.kv_buffer[host_layer_id]],\n                    src_indices=device_indices,\n                    dst_indices=host_indices,\n                    page_size=self.page_size,\n                )\n            else:\n                raise ValueError(\n                    \"Layer-sharded direct HiCache backup only supports \"\n                    f\"layer_first layout, got {self.layout}\"\n                )\n        else:\n            raise ValueError(\n                f\"Layer-sharded HiCache backup does not support IO backend: {io_backend}\"\n            )\n\n    def _resolve_device_transfer_buffers(self, device_pool):\n        if self.mtp_draft_device_pools:\n            return self.packed_device_data_ptrs, self.packed_device_kv_buffers\n        return device_pool.data_ptrs, device_pool.kv_buffer\n\n    def backup_from_device_all_layer(\n        self, device_pool, host_indices, device_indices, io_backend\n    ):\n        host_indices = self.maybe_dcp_kernel_indices(host_indices)\n        device_indices = self.maybe_dcp_kernel_indices(device_indices)","sourceCodeStart":384,"sourceCodeEnd":420,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/pool_host/mla.py#L384-L420","documentation":"When io_backend is \"direct\" and the device pool is layer-sharded, the per-layer backup only implements the layer_first host layout via transfer_kv_direct. Any other layout (page_first, page_first_kv_split, ...) hits this ValueError because a direct per-layer transfer would need scatter into a page-strided host buffer, which is not implemented.","triggerScenarios":"Calling backup_from_device_all_layer with io_backend=\"direct\" on a layer-sharded device pool while the host pool layout is page_first or another non-layer_first layout.","commonSituations":"Setting --hicache-io-backend direct (NIXL/GDS path) on a multi-GPU CP run where the host buffer was allocated page-first; or enabling page_first_direct expecting the direct backend to work per-layer.","solutions":["Use io_backend=\"kernel\" instead, which supports both layer_first and page_first","Switch the host layout to layer_first so the direct per-layer path applies","Avoid layer-sharded device pools (no CP) when you must use direct IO with page-first layouts"],"exampleFix":"# before\nserver_args.hicache_io_backend = \"direct\"   # layer-sharded pool + page_first\n# after\nserver_args.hicache_io_backend = \"kernel\"","handlingStrategy":"validation","validationCode":"if io_backend == \"direct\" and sharded and pool.layout != \"layer_first\":\n    io_backend = \"kernel\"  # direct per-layer needs layer_first","typeGuard":"def direct_per_layer_ok(pool) -> bool:\n    return pool.layout == \"layer_first\"","tryCatchPattern":"try:\n    pool.backup_from_device_all_layer(dp, hi, di, io_backend)\nexcept ValueError as e:\n    if \"direct HiCache backup only supports\" in str(e):\n        io_backend = \"kernel\"\n        pool.backup_from_device_all_layer(dp, hi, di, io_backend)\n    else:\n        raise","preventionTips":["Map direct IO to layer_first layout in configuration","Centralize backend/layout compatibility checks in one helper used by all transfer entry points"],"tags":["hicache","direct-io","layout","sglang"],"backgroundTag":"unsupported-configuration-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}