sgl-project/sglang · error · NotImplementedError

get_split_heads_page_buffer_meta requires layout='page_head'

Error message

get_split_heads_page_buffer_meta requires layout='page_head', which is not supported for models with head_dim != v_head_dim.

What it means

get_split_heads_page_buffer_meta only makes sense for layout='page_head', but this MHA host pool variant (for models with head_dim != v_head_dim) never supports page_head layout, so the method is unconditionally NotImplementedError. Split-heads page buffer metadata cannot be obtained for these models.

Source

Thrown at python/sglang/srt/mem_cache/pool_host/mha.py:1311

        else:
            raise ValueError(
                f"Unsupported IO backend for models with head_dim != v_head_dim: "
                f"{io_backend}; expected 'kernel' or 'direct'."
            )

    def get_data_page(self, index, flat: bool = True) -> torch.Tensor:
        raise self._flat_page_unsupported()

    def get_dummy_flat_data_page(self) -> torch.Tensor:
        raise self._flat_page_unsupported()

    def set_from_flat_data_page(self, index: int, data_page: torch.Tensor) -> None:
        raise self._flat_page_unsupported()

    def get_split_heads_page_buffer_meta(
        self, indices: torch.Tensor, split_factor: int
    ):
        raise NotImplementedError(
            "get_split_heads_page_buffer_meta requires layout='page_head', "
            "which is not supported for models with head_dim != v_head_dim."
        )

    def get_page_buffer_meta(self, indices):
        assert len(indices) % self.page_size == 0
        if self.layout not in ("page_first", "page_first_direct"):
            raise ValueError(
                f"Unsupported layout for models with head_dim != v_head_dim: "
                f"{self.layout}"
            )
        indices = indices.tolist()
        k_base_ptr = self.k_buffer.data_ptr()
        v_base_ptr = self.v_buffer.data_ptr()
        k_element_size = (
            self.layer_num
            * self.dtype.itemsize
            * self.page_size

View on GitHub (pinned to 0132848349)

Solutions

1. Do not call this method for models with head_dim != v_head_dim; branch on model/head-dim configuration
2. Use get_page_buffer_meta (which supports page_first / page_first_direct) instead
3. Gate the feature requiring split-heads metadata so it is skipped for these models

Example fix

// before
meta = pool.get_split_heads_page_buffer_meta(indices, split_factor)

// after
if pool.layout in ("page_first", "page_first_direct"):
    meta = pool.get_page_buffer_meta(indices)
else:
    meta = pool.get_split_heads_page_buffer_meta(indices, split_factor)

if not hasattr(pool, "get_split_heads_page_buffer_meta") or pool.layout not in ("page_head",):
    # fall back to page-buffer meta
    meta = pool.get_page_buffer_meta(indices)

def supports_split_heads(pool) -> bool:
    """True only for pools with page_head layout."""
    return getattr(pool, "layout", None) == "page_head"

try:
    meta = pool.get_split_heads_page_buffer_meta(indices, split_factor)
except NotImplementedError:
    meta = pool.get_page_buffer_meta(indices)

When it happens

Trigger: Calling get_split_heads_page_buffer_meta(indices, split_factor) on this pool class; any caller path (e.g. speculative decoding or hierarchical cache code) that assumes page_head layout exists for all pools.

Common situations: Generic code iterating over pool types calling this API unconditionally; new feature (e.g. split-heads speculative decode) enabled for a model whose head_dim != v_head_dim.

Related errors

• prefix-valid commit is unsupported for MXFP8 KV cache (it do
• move_kv_cache is not yet supported for MiniMaxSparseKVPool:
• Unsupported layout for models with head_dim != v_head_dim: {
• HiCache does not support Inkling MTP draft state yet.
• /v1/models ${response.status}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ae47da9301db3a1b. Report an issue: GitHub.