sgl-project/sglang · error · ValueError

Unsupported layout: {self.layout}

Error message

Unsupported layout: {self.layout}

What it means

The MHA host KV pool allocates its host-side buffer with a shape that depends on the storage layout; init_kv_buffer only knows how to build dims for implemented layouts (layer_first and page-first variants). An unknown layout string means no valid buffer shape can be computed, so construction fails.

Source

Thrown at python/sglang/srt/mem_cache/pool_host/mha.py:186

            dims = (
                2,
                self.page_num,
                self.layer_num,
                self.page_size,
                self.head_num,
                self.head_dim,
            )
        elif self.layout == "page_head":
            dims = (
                2,
                self.page_num,
                self.head_num,
                self.page_size,
                self.layer_num,
                self.head_dim,
            )
        else:
            raise ValueError(f"Unsupported layout: {self.layout}")
        self.token_stride_size = self.head_num * self.head_dim * self.dtype.itemsize
        self.layout_dim = self.token_stride_size * self.layer_num

        alloc_func = ALLOC_MEMORY_FUNCS[self.device_pool.device]
        buffer = alloc_func(
            dims,
            dtype=self.dtype,
            device=self.device,
            pin_memory=self.pin_memory,
            allocator=self.allocator,
        )
        return buffer

    def _init_write_back_staging_buffers(self):
        self.staging_page_capacity = 0
        self.staging_token_capacity = 0
        self.staging_k_buffer = None
        self.staging_v_buffer = None

View on GitHub (pinned to 0132848349)

Solutions

1. Print/inspect the layout string passed to the host pool constructor and fix it to one of the supported values ('layer_first', 'page_first', 'page_first_direct')
2. Do not set host KV layout manually unless needed; let SGLang derive it from server_args
3. If you added a new layout, extend init_kv_buffer with the corresponding dims construction (mirror the layer_first branch)
4. Upgrade SGLang / align fork with upstream layout constants

Example fix

# before
pool = MHAHostPool(..., layout="pagefirst")
# after
pool = MHAHostPool(..., layout="page_first")

SUPPORTED_LAYOUTS = ("layer_first", "page_first", "page_first_direct")
assert layout in SUPPORTED_LAYOUTS, f"unsupported host pool layout: {layout!r}"

def is_supported_layout(layout: str) -> bool:
    return layout in ("layer_first", "page_first", "page_first_direct")

When it happens

Trigger: Instantiating the MHA host pool (HierarchicalCache host KV buffer) with self.layout set to anything other than 'layer_first' / 'page_first' / 'page_first_direct' — e.g. a typo in the layout argument or a new layout value not handled at init.

Common situations: Custom ServerArgs / hierarchical-cache config passing a misspelled layout; code forks that introduce a new layout without updating init_kv_buffer; version mismatch where a layout constant was renamed.

Related errors

• Unsupported layout for models with head_dim != v_head_dim: {
• Unsupported layout for models with head_dim != v_head_dim: {
• v_cache must be provided
• k_cache can only be None when only_qv=True
• Invalid mode: {mode}, must be one of 'write', 'read', 'skip'

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/593494569ce62bc4. Report an issue: GitHub.