{"record":{"id":"0660b3bc1c17ad3b","repo":"sgl-project/sglang","slug":"unsupported-layout-for-models-with-head-dim-v-h-0660b3","errorCode":null,"errorMessage":"Unsupported layout for models with head_dim != v_head_dim and io_backend='kernel': {self.layout}; expected 'page_first'.","messagePattern":"Unsupported layout for models with head_dim != v_head_dim and io_backend='kernel': (.+?); expected 'page_first'\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/pool_host/mha.py","lineNumber":1170,"sourceCode":"        host_indices,\n        device_indices,\n        layer_id,\n        io_backend,\n        *,\n        is_draft: bool = False,\n    ):\n        if self.device_pool is not None:\n            if not is_draft and not self._is_device_layer_owned(device_pool, layer_id):\n                return\n            # MTP draft layers do not participate in CP layer sharding.\n            host_layer_id = layer_id if is_draft else self._host_layer_index(layer_id)\n            device_layer_id = 0 if is_draft else layer_id\n        else:\n            host_layer_id = device_layer_id = layer_id\n\n        if io_backend == \"kernel\":\n            if self.layout != \"page_first\":\n                raise ValueError(\n                    f\"Unsupported layout for models with head_dim != v_head_dim \"\n                    f\"and io_backend='kernel': {self.layout}; expected 'page_first'.\"\n                )\n            transfer_kv_per_layer_mla_pf_lf(\n                src=self.k_buffer,\n                dst=device_pool.k_buffer[device_layer_id],\n                src_indices=host_indices,\n                dst_indices=device_indices,\n                layer_id=host_layer_id,\n                item_size=self._k_token_stride_size(),\n                src_layout_dim=self._k_layout_dim(),\n            )\n            transfer_kv_per_layer_mla_pf_lf(\n                src=self.v_buffer,\n                dst=device_pool.v_buffer[device_layer_id],\n                src_indices=host_indices,\n                dst_indices=device_indices,\n                layer_id=host_layer_id,","sourceCodeStart":1152,"sourceCodeEnd":1188,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/pool_host/mha.py#L1152-L1188","documentation":"For MLA models with head_dim != v_head_dim, load_to_device_per_layer with io_backend='kernel' uses transfer_kv_per_layer_mla_pf_lf which requires the host buffer in 'page_first' layout so its kernel indexing is valid.","triggerScenarios":"Calling load_to_device_per_layer(..., io_backend='kernel') on a head_dim!=v_head_dim host pool whose layout is 'page_first_direct' or anything else.","commonSituations":"Setting hicache io_backend to 'kernel' while the host pool was allocated with page_first_direct layout (mismatched config); mixing direct-layout storage files with kernel backend.","solutions":["Set the host pool layout to 'page_first' when using io_backend='kernel'","Or switch io_backend to 'direct' to use a page_first_direct pool","Align hicache_io_backend and host pool layout in server args"],"exampleFix":"# before\npool.load_to_device_per_layer(dev, layer_id=0, host_indices=h, device_indices=d, io_backend=\"kernel\")  # layout == \"page_first_direct\"\n# after\npool.load_to_device_per_layer(dev, layer_id=0, host_indices=h, device_indices=d, io_backend=\"direct\")","handlingStrategy":"validation","validationCode":"if io_backend == \"kernel\":\n    assert pool.layout == \"page_first\", \"kernel backend requires page_first for asymmetric head dims\"","typeGuard":"def kernel_backend_compatible(pool, io_backend: bool) -> bool:\n    return io_backend != \"kernel\" or pool.layout == \"page_first\"","tryCatchPattern":"try:\n    pool.load_to_device_per_layer(...)\nexcept ValueError as e:\n    if \"expected 'page_first'\" in str(e):\n        rebuild pool with layout=\"page_first\", or retry with io_backend=\"direct\"\n    raise","preventionTips":["Pair kernel backend with page_first layout","Configure both from server_args so they stay consistent"],"tags":["sglang","mla","hicache","io-backend","layout"],"backgroundTag":"layout-backend-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}