{"record":{"id":"ae47da9301db3a1b","repo":"sgl-project/sglang","slug":"get-split-heads-page-buffer-meta-requires-layout","errorCode":null,"errorMessage":"get_split_heads_page_buffer_meta requires layout='page_head', which is not supported for models with head_dim != v_head_dim.","messagePattern":"get_split_heads_page_buffer_meta requires layout='page_head', which is not supported for models with head_dim != v_head_dim\\.","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/pool_host/mha.py","lineNumber":1311,"sourceCode":"        else:\n            raise ValueError(\n                f\"Unsupported IO backend for models with head_dim != v_head_dim: \"\n                f\"{io_backend}; expected 'kernel' or 'direct'.\"\n            )\n\n    def get_data_page(self, index, flat: bool = True) -> torch.Tensor:\n        raise self._flat_page_unsupported()\n\n    def get_dummy_flat_data_page(self) -> torch.Tensor:\n        raise self._flat_page_unsupported()\n\n    def set_from_flat_data_page(self, index: int, data_page: torch.Tensor) -> None:\n        raise self._flat_page_unsupported()\n\n    def get_split_heads_page_buffer_meta(\n        self, indices: torch.Tensor, split_factor: int\n    ):\n        raise NotImplementedError(\n            \"get_split_heads_page_buffer_meta requires layout='page_head', \"\n            \"which is not supported for models with head_dim != v_head_dim.\"\n        )\n\n    def get_page_buffer_meta(self, indices):\n        assert len(indices) % self.page_size == 0\n        if self.layout not in (\"page_first\", \"page_first_direct\"):\n            raise ValueError(\n                f\"Unsupported layout for models with head_dim != v_head_dim: \"\n                f\"{self.layout}\"\n            )\n        indices = indices.tolist()\n        k_base_ptr = self.k_buffer.data_ptr()\n        v_base_ptr = self.v_buffer.data_ptr()\n        k_element_size = (\n            self.layer_num\n            * self.dtype.itemsize\n            * self.page_size","sourceCodeStart":1293,"sourceCodeEnd":1329,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/pool_host/mha.py#L1293-L1329","documentation":"get_split_heads_page_buffer_meta only makes sense for layout='page_head', but this MHA host pool variant (for models with head_dim != v_head_dim) never supports page_head layout, so the method is unconditionally NotImplementedError. Split-heads page buffer metadata cannot be obtained for these models.","triggerScenarios":"Calling get_split_heads_page_buffer_meta(indices, split_factor) on this pool class; any caller path (e.g. speculative decoding or hierarchical cache code) that assumes page_head layout exists for all pools.","commonSituations":"Generic code iterating over pool types calling this API unconditionally; new feature (e.g. split-heads speculative decode) enabled for a model whose head_dim != v_head_dim.","solutions":["Do not call this method for models with head_dim != v_head_dim; branch on model/head-dim configuration","Use get_page_buffer_meta (which supports page_first / page_first_direct) instead","Gate the feature requiring split-heads metadata so it is skipped for these models"],"exampleFix":"// before\nmeta = pool.get_split_heads_page_buffer_meta(indices, split_factor)\n\n// after\nif pool.layout in (\"page_first\", \"page_first_direct\"):\n    meta = pool.get_page_buffer_meta(indices)\nelse:\n    meta = pool.get_split_heads_page_buffer_meta(indices, split_factor)","handlingStrategy":"type-guard","validationCode":"if not hasattr(pool, \"get_split_heads_page_buffer_meta\") or pool.layout not in (\"page_head\",):\n    # fall back to page-buffer meta\n    meta = pool.get_page_buffer_meta(indices)","typeGuard":"def supports_split_heads(pool) -> bool:\n    \"\"\"True only for pools with page_head layout.\"\"\"\n    return getattr(pool, \"layout\", None) == \"page_head\"","tryCatchPattern":"try:\n    meta = pool.get_split_heads_page_buffer_meta(indices, split_factor)\nexcept NotImplementedError:\n    meta = pool.get_page_buffer_meta(indices)","preventionTips":["Branch on pool.layout/model head dims before calling layout-specific APIs","Treat NotImplementedError from these pools as a signal to use the generic path"],"tags":["sglang","hicache","not-implemented","kv-cache"],"backgroundTag":"unsupported-feature-for-model","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}