sgl-project/sglang · error · ValueError

Unsupported layout for models with head_dim != v_head_dim an

Error message

Unsupported layout for models with head_dim != v_head_dim and io_backend='direct': {self.layout}; expected 'page_first_direct'.

What it means

The 'direct' IO backend for head_dim != v_head_dim models uses transfer_kv_per_layer_direct_pf_lf, which requires host buffers in 'page_first_direct' layout (one contiguous block per page). Other layouts make the pointer math invalid.

Source

Thrown at python/sglang/srt/mem_cache/pool_host/mha.py:1194

                dst=device_pool.k_buffer[device_layer_id],
                src_indices=host_indices,
                dst_indices=device_indices,
                layer_id=host_layer_id,
                item_size=self._k_token_stride_size(),
                src_layout_dim=self._k_layout_dim(),
            )
            transfer_kv_per_layer_mla_pf_lf(
                src=self.v_buffer,
                dst=device_pool.v_buffer[device_layer_id],
                src_indices=host_indices,
                dst_indices=device_indices,
                layer_id=host_layer_id,
                item_size=self._v_token_stride_size(),
                src_layout_dim=self._v_layout_dim(),
            )
        elif io_backend == "direct":
            if self.layout != "page_first_direct":
                raise ValueError(
                    f"Unsupported layout for models with head_dim != v_head_dim "
                    f"and io_backend='direct': {self.layout}; expected "
                    "'page_first_direct'."
                )
            transfer_kv_per_layer_direct_pf_lf(
                src_ptrs=[self.k_buffer],
                dst_ptrs=[device_pool.k_buffer[device_layer_id]],
                src_indices=host_indices,
                dst_indices=device_indices,
                layer_id=host_layer_id,
                page_size=self.page_size,
            )
            transfer_kv_per_layer_direct_pf_lf(
                src_ptrs=[self.v_buffer],
                dst_ptrs=[device_pool.v_buffer[device_layer_id]],
                src_indices=host_indices,
                dst_indices=device_indices,
                layer_id=host_layer_id,

View on GitHub (pinned to 0132848349)

Solutions

1. Set the host pool layout to 'page_first_direct' when using io_backend='direct'
2. Or use io_backend='kernel' with the default 'page_first' layout
3. Recreate the host pool after changing layout — buffers are allocated per layout

Example fix

# before
pool = MLATokenToKVPoolHost(..., layout="page_first")
pool.load_to_device_per_layer(dev, ..., io_backend="direct")
# after
pool = MLATokenToKVPoolHost(..., layout="page_first_direct")
pool.load_to_device_per_layer(dev, ..., io_backend="direct")

if io_backend == "direct":
    assert pool.layout == "page_first_direct", "direct backend requires page_first_direct for asymmetric head dims"

def direct_backend_compatible(pool, io_backend: str) -> bool:
    return io_backend != "direct" or pool.layout == "page_first_direct"

try:
    pool.load_to_device_per_layer(...)
except ValueError as e:
    if "expected 'page_first_direct'" in str(e):
        retry with io_backend="kernel" on a page_first pool
    raise

When it happens

Trigger: Calling load_to_device_per_layer(..., io_backend='direct') on a head_dim!=v_head_dim pool with layout 'page_first' or 'layer_first'.

Common situations: Default page_first layout combined with hicache_io_backend='direct'; switching backends without re-allocating the host pool.

Related errors

• Unsupported layout for models with head_dim != v_head_dim an
• Unsupported V4 paged host layout/backend: {self.layout}/{io_
• Unsupported IO backend for models with head_dim != v_head_di
• Unsupported layout: {self.layout}
• Unsupported IO backend: {io_backend}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ece7835da83ee7a8. Report an issue: GitHub.