{"record":{"id":"3a8383100ca5c288","repo":"sgl-project/sglang","slug":"unsupported-layout-for-models-with-head-dim-v-h","errorCode":null,"errorMessage":"Unsupported layout for models with head_dim != v_head_dim: {self.layout}; expected 'page_first' or 'page_first_direct'.","messagePattern":"Unsupported layout for models with head_dim != v_head_dim: (.+?); expected 'page_first' or 'page_first_direct'\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/pool_host/mha.py","lineNumber":1103,"sourceCode":"            k_dims = (self.size, self.layer_num, self.head_num, self.head_dim)\n            v_dims = (self.size, self.layer_num, self.head_num, self.v_head_dim)\n        elif self.layout == \"page_first_direct\":\n            k_dims = (\n                self.page_num,\n                self.layer_num,\n                self.page_size,\n                self.head_num,\n                self.head_dim,\n            )\n            v_dims = (\n                self.page_num,\n                self.layer_num,\n                self.page_size,\n                self.head_num,\n                self.v_head_dim,\n            )\n        else:\n            raise ValueError(\n                f\"Unsupported layout for models with head_dim != v_head_dim: \"\n                f\"{self.layout}; expected 'page_first' or 'page_first_direct'.\"\n            )\n\n        # token_stride_size / layout_dim are intentionally NOT set: K and V\n        # have different strides, so any caller that reaches for a single\n        # shared stride is a bug. Such callers will fail loudly with\n        # AttributeError rather than silently use the K stride for V copies.\n\n        alloc_func = ALLOC_MEMORY_FUNCS[self.device_pool.device]\n        k_buffer = alloc_func(\n            k_dims,\n            dtype=self.dtype,\n            device=self.device,\n            pin_memory=self.pin_memory,\n            allocator=self.allocator,\n        )\n        v_buffer = alloc_func(","sourceCodeStart":1085,"sourceCodeEnd":1121,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/pool_host/mha.py#L1085-L1121","documentation":"In the MLA variant with head_dim != v_head_dim, K and V buffers need different shapes/strides; init_kv_buffer only knows how to allocate them for 'page_first' and 'page_first_direct'. Any other layout (notably 'layer_first') cannot represent the asymmetric K/V dims.","triggerScenarios":"Constructing the head_dim!=v_head_dim host pool class with layout='layer_first' or an unknown value; init_kv_buffer then refuses to allocate k_buffer/v_buffer.","commonSituations":"Running MLA-style or hybrid models with a legacy layer-first host layout config; carrying over configs from MHA models to MLA models.","solutions":["Use layout='page_first' (recommended, works with kernel backend) or 'page_first_direct'","Remove any explicit layer_first host-layout override in server args / hicache config","Check model config: if head_dim == v_head_dim you can use the standard MHA pool with more layouts"],"exampleFix":"# before\npool = MLATokenToKVPoolHost(..., layout=\"layer_first\")\n# after\npool = MLATokenToKVPoolHost(..., layout=\"page_first\")","handlingStrategy":"validation","validationCode":"if getattr(pool, \"head_dim\", None) != getattr(pool, \"v_head_dim\", None):\n    assert pool.layout in (\"page_first\", \"page_first_direct\"), \"asymmetric head dims require page-first layouts\"","typeGuard":"def mla_pool_layout_ok(pool) -> bool:\n    if pool.head_dim == pool.v_head_dim:\n        return True\n    return pool.layout in (\"page_first\", \"page_first_direct\")","tryCatchPattern":null,"preventionTips":["Default MLA host pools to page_first","Don't reuse MHA layer_first configs for MLA models"],"tags":["sglang","mla","kv-cache","layout","host-pool"],"backgroundTag":"unsupported-layout-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}