sgl-project/sglang · error · RuntimeError

Mamba state layouts differ between prefill and decode (src i

Error message

Mamba state layouts differ between prefill and decode (src item_lens={src_item_lens}, dst item_lens={dst_item_lens}) and no per-tensor dim metadata is available to reslice. With --enable-unified-memory, prefill and decode must both enable it and use equal attention TP sizes.

What it means

In maybe_send_extra, mamba state tensors whose item_lens differ between prefill and decode cannot be resliced because no per-tensor dim metadata exists; the transfer is aborted with guidance to match TP sizes and flags.

Source

Thrown at python/sglang/srt/disaggregation/mooncake/conn.py:1290

                    else []
                )
                dst_state_layer_ids = (
                    target_rank_registration_info.dst_state_layer_ids[i]
                    if i < len(target_rank_registration_info.dst_state_layer_ids)
                    else []
                )
            else:
                dst_data_ptrs, dst_item_lens, dst_dim_per_tensor = [], [], []
                dst_state_layer_ids = []
            dst_indices = (
                req.dst_state_indices[i] if i < len(req.dst_state_indices) else []
            )

            if st == StateType.MAMBA:
                if (not src_dim_per_tensor or not dst_dim_per_tensor) and list(
                    src_item_lens
                ) != list(dst_item_lens):
                    raise RuntimeError(
                        "Mamba state layouts differ between prefill and decode "
                        f"(src item_lens={src_item_lens}, dst item_lens="
                        f"{dst_item_lens}) and no per-tensor dim metadata is "
                        "available to reslice. With --enable-unified-memory, "
                        "prefill and decode must both enable it and use equal "
                        "attention TP sizes."
                    )
                if (
                    target_rank_registration_info is not None
                    and self.attn_tp_size
                    != target_rank_registration_info.dst_attn_tp_size
                ):
                    rc = (
                        self._send_mamba_state_slice(
                            req,
                            indices,
                            src_data_ptrs,
                            src_item_lens,

View on GitHub (pinned to 0132848349)

Solutions

1. Equalize attention TP sizes on prefill and decode
2. Enable --enable-unified-memory on both sides so dim metadata is available
3. Upgrade both instances to the same sglang version so registration carries per-tensor dims

When it happens

Trigger: Hybrid model PD transfer where src and dst mamba item_lens differ AND src_dim_per_tensor/dst_dim_per_tensor are missing (older registration payloads).

Common situations: Mixed sglang versions between prefill and decode, mismatched --enable-unified-memory, or heterogeneous attention TP.

Related errors

• --enable-unified-memory does not support different prefill /
• PD KV layout mismatch on the whole-envelope path: prefill ha
• PD Disaggregation does NOT support PD different TP sizes for
• PD disagg: heterogeneous TP not supported for MiniMax sparse
• `mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim}).

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/31cb6847e4dd7ab7. Report an issue: GitHub.