sgl-project/sglang · error · ValueError

Unsupported layout for models with head_dim != v_head_dim: {

Error message

Unsupported layout for models with head_dim != v_head_dim: {self.layout}

What it means

get_page_buffer_meta on the head_dim != v_head_dim MHA host pool only supports 'page_first' and 'page_first_direct' layouts. The pool was constructed with some other layout (e.g. 'layer_first' or 'page_head'), so page-buffer metadata cannot be computed.

Source

Thrown at python/sglang/srt/mem_cache/pool_host/mha.py:1319

    def get_dummy_flat_data_page(self) -> torch.Tensor:
        raise self._flat_page_unsupported()

    def set_from_flat_data_page(self, index: int, data_page: torch.Tensor) -> None:
        raise self._flat_page_unsupported()

    def get_split_heads_page_buffer_meta(
        self, indices: torch.Tensor, split_factor: int
    ):
        raise NotImplementedError(
            "get_split_heads_page_buffer_meta requires layout='page_head', "
            "which is not supported for models with head_dim != v_head_dim."
        )

    def get_page_buffer_meta(self, indices):
        assert len(indices) % self.page_size == 0
        if self.layout not in ("page_first", "page_first_direct"):
            raise ValueError(
                f"Unsupported layout for models with head_dim != v_head_dim: "
                f"{self.layout}"
            )
        indices = indices.tolist()
        k_base_ptr = self.k_buffer.data_ptr()
        v_base_ptr = self.v_buffer.data_ptr()
        k_element_size = (
            self.layer_num
            * self.dtype.itemsize
            * self.page_size
            * self.head_num
            * self.head_dim
        )
        v_element_size = (
            self.layer_num
            * self.dtype.itemsize
            * self.page_size
            * self.head_num

View on GitHub (pinned to 0132848349)

Solutions

Construct the host pool with layout='page_first' (or 'page_first_direct')
Check the server arg / pool allocator that selects layout and align it
Avoid calling get_page_buffer_meta for layer-first pools; use the per-layer transfer APIs instead

Example fix

// before
MHATokenToKVPoolHost(..., layout="layer_first")  # later: pool.get_page_buffer_meta(idx) fails

// after
MHATokenToKVPoolHost(..., layout="page_first")

Defensive patterns

Strategy: validation

Validate before calling

if pool.layout not in ("page_first", "page_first_direct"):
    raise ConfigError(f"page-buffer meta requires page_first layouts, got {pool.layout}")

Type guard

def has_page_buffer_meta(pool) -> bool:
    return getattr(pool, "layout", None) in ("page_first", "page_first_direct")

Try / catch

try:
    meta = pool.get_page_buffer_meta(indices)
except ValueError as e:
    if "Unsupported layout" in str(e):
        meta = None  # use per-layer transfer APIs instead
    else:
        raise

Prevention

Choose page_first layout when zero-copy page transfers are needed
Assert layout invariants right after pool construction

When it happens

Trigger: Calling get_page_buffer_meta(indices) when self.layout is not 'page_first' or 'page_first_direct'.

Common situations: HiCache configured with a layer-first host layout (common on some backends) while code needs page-buffer meta for zero-copy page transfers; version changes altering default layout.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bbba937c54afad3c. Report an issue: GitHub.