sgl-project/sglang · error · ValueError

Layer-sharded HiCache backup does not support layout: {self.

Error message

Layer-sharded HiCache backup does not support layout: {self.layout}

What it means

MLAHiCachePool._backup_from_device_per_layer only knows how to write layer_first and page_first host layouts when the device KV pool is layer-sharded (e.g. with context parallelism). This error fires when the pool's layout is anything else (such as page_first_kv_split) and a per-layer D2H backup is attempted, because no code path exists for that layout combination.

Source

Thrown at python/sglang/srt/mem_cache/pool_host/mla.py:389

                        dst_indices=host_indices,
                        item_size=self.token_stride_size,
                    )
            elif self.layout == "page_first":
                if self.can_use_jit:
                    jit_transfer_hicache_one_layer_mla(
                        cache_dst=self.data_refs[host_layer_id],
                        cache_src=device_pool.kv_buffer[device_layer_id],
                        indices_dst=host_indices,
                        indices_src=device_indices,
                        element_dim=self.kv_cache_dim,
                    )
                else:
                    raise ValueError(
                        "Layer-sharded MLA HiCache backup with page_first layout "
                        "requires the JIT one-layer kernel."
                    )
            else:
                raise ValueError(
                    f"Layer-sharded HiCache backup does not support layout: {self.layout}"
                )
        elif io_backend == "direct":
            if self.layout == "layer_first":
                transfer_kv_direct(
                    src_layers=[device_pool.kv_buffer[device_layer_id]],
                    dst_layers=[self.kv_buffer[host_layer_id]],
                    src_indices=device_indices,
                    dst_indices=host_indices,
                    page_size=self.page_size,
                )
            else:
                raise ValueError(
                    "Layer-sharded direct HiCache backup only supports "
                    f"layer_first layout, got {self.layout}"
                )
        else:
            raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

1. Switch the host HiCache layout to layer_first (or page_first with the JIT kernel enabled) so the per-layer path is supported
2. Disable layer sharding of the device pool (e.g. drop context parallelism / CP>1) so the all-layer path handles the layout
3. If you control the code, extend _backup_from_device_per_layer with a branch for the missing layout

Example fix

# before
server_args.hicache_layout = "page_first_kv_split"  # with CP layer sharding
# after
server_args.hicache_layout = "layer_first"

SUPPORTED_PER_LAYER = {"layer_first", "page_first"}
if pool._is_device_layer_sharded(device_pool) and pool.layout not in SUPPORTED_PER_LAYER:
    raise ConfigError(f"layout {pool.layout} unsupported for layer-sharded backup")

def supports_layer_sharded_backup(pool) -> bool:
    return pool.layout in ("layer_first", "page_first")

try:
    pool.backup_from_device_all_layer(...)
except ValueError as e:
    if "Layer-sharded" in str(e):
        logger.error("layout/io-backend combo unsupported; falling back to disabled HiCache")
        disable_hicache()
    else:
        raise

When it happens

Trigger: Running a model with a layer-sharded device pool (CP) plus a host HiCache layout other than layer_first/page_first, causing backup_from_device_all_layer to loop over owned layers and call _backup_from_device_per_layer, which falls through the layout if/elif chain.

Common situations: Using --hicache with page_first_kv_split layout (Ascend-style KV-split buffers) together with context parallelism, or a new layout added to the class without extending this branch.

Related errors

• Unsupported layout for models with head_dim != v_head_dim an
• Unsupported layout for models with head_dim != v_head_dim an
• Unsupported layout: {self.layout}
• trtllm_mla cannot serve decode context parallelism with spec
• trtllm_mla does not forward the cyclic DCP metadata to its d

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/49b59457995ca050. Report an issue: GitHub.