sgl-project/sglang · error · ValueError

Layer-sharded DSA indexer HiCache backup with page_first lay

Error message

Layer-sharded DSA indexer HiCache backup with page_first layout is not supported without a per-layer LF->PF kernel.

What it means

Backing up (device->host) a layer-sharded DSA indexer per-layer with page_first layout would require converting layer-first device data to page-first host layout per layer, and no such kernel exists, so the operation is explicitly rejected.

Source

Thrown at python/sglang/srt/mem_cache/pool_host/dsa.py:312

        # MTP draft layers do not participate in CP layer sharding.
        host_layer_id = layer_id if is_draft else self._host_layer_index(layer_id)
        device_layer_id = 0 if is_draft else layer_id

        host_page_indices, device_page_indices = self._get_indexer_page_indices(
            host_indices, device_indices
        )
        use_kernel = io_backend == "kernel" and self.indexer_page_stride_size % 8 == 0
        if use_kernel:
            if self.layout == "layer_first":
                transfer_kv_per_layer_mla(
                    src=device_pool.index_k_with_scale_buffer[device_layer_id],
                    dst=self.index_k_with_scale_buffer[host_layer_id],
                    src_indices=device_page_indices,
                    dst_indices=host_page_indices,
                    item_size=self.indexer_page_stride_size,
                )
            elif self.layout == "page_first":
                raise ValueError(
                    "Layer-sharded DSA indexer HiCache backup with page_first "
                    "layout is not supported without a per-layer LF->PF kernel."
                )
            else:
                raise ValueError(f"Unsupported layout: {self.layout}")
        elif io_backend == "direct":
            if self.layout == "layer_first":
                transfer_kv_direct(
                    src_layers=[device_pool.index_k_with_scale_buffer[device_layer_id]],
                    dst_layers=[self.index_k_with_scale_buffer[host_layer_id]],
                    src_indices=device_page_indices,
                    dst_indices=host_page_indices,
                    page_size=1,
                )
            else:
                raise ValueError(
                    "Layer-sharded direct DSA indexer backup only supports "
                    f"layer_first layout, got {self.layout}"

View on GitHub (pinned to 0132848349)

Solutions

1. Switch host layout to 'layer_first'
2. Or avoid layer-sharded device pools with the DSA HiCache backup path
3. Track upstream for a per-layer LF->PF kernel

Example fix

# before
layout='page_first'  # + layer-sharded device pool
# after
layout='layer_first'

if layer_sharded and io_backend == "kernel":
    assert layout == "layer_first"

def sharded_backup_supported(layout: str, io_backend: str) -> bool:
    return layout == "layer_first"

When it happens

Trigger: backup_from_device_all_layer on a layer-sharded device pool where layout='page_first', with the kernel IO backend.

Common situations: Enabling layer sharding (e.g. PP/layer-parallel device pools) together with page_first host layout and hierarchical cache backup.

Related errors

• Unsupported layout: {self.layout}
• Layer-sharded direct DSA indexer backup only supports layer_
• Not a canonical UMMA_MN Layout: Expected profile failure.
• Not a canonical UMMA_MN Layout: Expected stride failure.
• SWIZZLE_128B_BASE32B is invalid for Major-K

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6a39c170a60831fb. Report an issue: GitHub.