sgl-project/sglang · error · ValueError

Mamba storage zero-copy requires page_first layout, got {sel

Error message

Mamba storage zero-copy requires page_first layout, got {self.layout}

What it means

The Mamba host storage's zero-copy page-buffer metadata API only works when the host pool was allocated in a page-first layout, because only then is each page slot in the temporal/conv state buffers directly addressable as contiguous memory. If the pool was created with layer_first layout, per-page pointers cannot be computed, so the API refuses.

Source

Thrown at python/sglang/srt/mem_cache/pool_host/mamba.py:542

    ) -> None:
        flat_bytes = data_page.contiguous().view(torch.uint8).reshape(-1)
        start = 0
        for tensor in self._iter_page_tensors(index):
            num_bytes = tensor.numel() * tensor.element_size()
            tensor_bytes = flat_bytes[start : start + num_bytes]
            start += num_bytes
            restored = tensor_bytes.view(dtype=tensor.dtype).reshape(tensor.shape)
            tensor.copy_(restored)

    def get_page_buffer_meta(self, indices):
        """Meta data for zero-copy storage I/O.

        Only page-first layouts are supported for mamba storage zero-copy because
        each page slot in temporal/conv buffers is directly addressable.
        """
        assert len(indices) % self.page_size == 0
        if self.layout not in ["page_first", "page_first_direct"]:
            raise ValueError(
                f"Mamba storage zero-copy requires page_first layout, got {self.layout}"
            )
        indices = indices.tolist()
        ptr_list = []
        element_size_list = []

        # Compute base pointers once; each page pointer is offset from these bases.
        temporal_base_ptr = self.temporal_buffer.data_ptr()
        conv_base_ptrs = [buf.data_ptr() for buf in self.conv_buffer]
        # Component sizes are constant across pages, so precompute once as well.
        temporal_element_size = (
            self.page_size
            * self.num_mamba_layers
            * self.temporal_dtype.itemsize
            * self.temporal_state_elem_size
        )
        conv_element_sizes = [
            (

View on GitHub (pinned to 0132848349)

Solutions

Configure the host pool / hierarchical cache so the Mamba host storage uses page_first layout (enable the page-first host layout option in ServerArgs or the pool constructor)
If using a transfer API that requires get_page_buffer_meta, switch to the non-zero-copy path that works with layer_first
Upgrade SGLang if your version defaulted Mamba host layout incorrectly for zero-copy offload
Ensure indices length is a multiple of page_size as well, since the same API asserts on that

Example fix

# before
host_pool = MambaHostStorage(..., layout="layer_first")
meta = host_pool.get_page_buffer_meta(indices)  # ValueError
# after
host_pool = MambaHostStorage(..., layout="page_first")
meta = host_pool.get_page_buffer_meta(indices)

Defensive patterns

Strategy: validation

Validate before calling

assert host_storage.layout in ("page_first", "page_first_direct"), (
    f"get_page_buffer_meta requires page_first layout, got {host_storage.layout}")
assert len(indices) % host_storage.page_size == 0

Type guard

def supports_zero_copy(storage) -> bool:
    return getattr(storage, "layout", "") in ("page_first", "page_first_direct")

Prevention

Construct Mamba host storage with page_first layout whenever zero-copy transfer is intended
Centralize layout constants instead of passing raw strings
Assert page-size-aligned indices before calling

When it happens

Trigger: Calling MambaHostStorage.get_page_buffer_meta(indices) while self.layout is not 'page_first' or 'page_first_direct' — typically when the hierarchical cache was configured without the page-first host layout option (e.g. hierarchical cache for Mamba with layer-first host allocation).

Common situations: Enabling hierarchical cache / zero-copy offload on a hybrid Mamba model without setting the host pool layout to page-first; older configs that defaulted Mamba host pools to layer_first; custom code calling the zero-copy transfer API directly.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2f7ab6b5e0644056. Report an issue: GitHub.