{"record":{"id":"ece7835da83ee7a8","repo":"sgl-project/sglang","slug":"unsupported-layout-for-models-with-head-dim-v-h-ece783","errorCode":null,"errorMessage":"Unsupported layout for models with head_dim != v_head_dim and io_backend='direct': {self.layout}; expected 'page_first_direct'.","messagePattern":"Unsupported layout for models with head_dim != v_head_dim and io_backend='direct': (.+?); expected 'page_first_direct'\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/pool_host/mha.py","lineNumber":1194,"sourceCode":"                dst=device_pool.k_buffer[device_layer_id],\n                src_indices=host_indices,\n                dst_indices=device_indices,\n                layer_id=host_layer_id,\n                item_size=self._k_token_stride_size(),\n                src_layout_dim=self._k_layout_dim(),\n            )\n            transfer_kv_per_layer_mla_pf_lf(\n                src=self.v_buffer,\n                dst=device_pool.v_buffer[device_layer_id],\n                src_indices=host_indices,\n                dst_indices=device_indices,\n                layer_id=host_layer_id,\n                item_size=self._v_token_stride_size(),\n                src_layout_dim=self._v_layout_dim(),\n            )\n        elif io_backend == \"direct\":\n            if self.layout != \"page_first_direct\":\n                raise ValueError(\n                    f\"Unsupported layout for models with head_dim != v_head_dim \"\n                    f\"and io_backend='direct': {self.layout}; expected \"\n                    \"'page_first_direct'.\"\n                )\n            transfer_kv_per_layer_direct_pf_lf(\n                src_ptrs=[self.k_buffer],\n                dst_ptrs=[device_pool.k_buffer[device_layer_id]],\n                src_indices=host_indices,\n                dst_indices=device_indices,\n                layer_id=host_layer_id,\n                page_size=self.page_size,\n            )\n            transfer_kv_per_layer_direct_pf_lf(\n                src_ptrs=[self.v_buffer],\n                dst_ptrs=[device_pool.v_buffer[device_layer_id]],\n                src_indices=host_indices,\n                dst_indices=device_indices,\n                layer_id=host_layer_id,","sourceCodeStart":1176,"sourceCodeEnd":1212,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/pool_host/mha.py#L1176-L1212","documentation":"The 'direct' IO backend for head_dim != v_head_dim models uses transfer_kv_per_layer_direct_pf_lf, which requires host buffers in 'page_first_direct' layout (one contiguous block per page). Other layouts make the pointer math invalid.","triggerScenarios":"Calling load_to_device_per_layer(..., io_backend='direct') on a head_dim!=v_head_dim pool with layout 'page_first' or 'layer_first'.","commonSituations":"Default page_first layout combined with hicache_io_backend='direct'; switching backends without re-allocating the host pool.","solutions":["Set the host pool layout to 'page_first_direct' when using io_backend='direct'","Or use io_backend='kernel' with the default 'page_first' layout","Recreate the host pool after changing layout — buffers are allocated per layout"],"exampleFix":"# before\npool = MLATokenToKVPoolHost(..., layout=\"page_first\")\npool.load_to_device_per_layer(dev, ..., io_backend=\"direct\")\n# after\npool = MLATokenToKVPoolHost(..., layout=\"page_first_direct\")\npool.load_to_device_per_layer(dev, ..., io_backend=\"direct\")","handlingStrategy":"validation","validationCode":"if io_backend == \"direct\":\n    assert pool.layout == \"page_first_direct\", \"direct backend requires page_first_direct for asymmetric head dims\"","typeGuard":"def direct_backend_compatible(pool, io_backend: str) -> bool:\n    return io_backend != \"direct\" or pool.layout == \"page_first_direct\"","tryCatchPattern":"try:\n    pool.load_to_device_per_layer(...)\nexcept ValueError as e:\n    if \"expected 'page_first_direct'\" in str(e):\n        retry with io_backend=\"kernel\" on a page_first pool\n    raise","preventionTips":["Pair direct backend with page_first_direct layout","Recreate the pool when switching layout/backend"],"tags":["sglang","mla","hicache","io-backend","layout"],"backgroundTag":"layout-backend-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}