sgl-project/sglang · error · RuntimeError

PD state transfer does not support TP-mismatched non-MLA SWA

Error message

PD state transfer does not support TP-mismatched non-MLA SWA models (prefill_tp_size={self.attn_tp_size}, decode_tp_size={peer_info.decode_tp_size})

What it means

For SWA (sliding-window attention) state transfer on non-MLA backends, the sender requires prefill TP size to equal decode TP size. The SWA state layout depends on TP head sharding, and the transfer path has no slicing support for it unless MLA is used.

Source

Thrown at python/sglang/srt/disaggregation/mori/conn.py:1373

        return statuses

    def _send_swa_dsa_state(
        self,
        peer_info: KVArgsRegisterInfo,
        src_state_indices: npt.NDArray[np.int32],
        dst_state_indices: npt.NDArray[np.int32],
        src_state_mem_descs: List[MemoryDesc],
        src_state_item_lens: List[int],
        dst_state_mem_descs: List[MemoryDesc],
        state_type: str,
    ) -> List[TransferStatus]:
        # TP mismatch check for non-MLA SWA
        if (
            state_type == "swa"
            and not self.is_mla_backend
            and peer_info.decode_tp_size != self.attn_tp_size
        ):
            raise RuntimeError(
                f"PD state transfer does not support TP-mismatched non-MLA SWA models "
                f"(prefill_tp_size={self.attn_tp_size}, decode_tp_size={peer_info.decode_tp_size})"
            )
        if state_type == "minimax_index_k":
            if self.pp_size is not None and self.pp_size > 1:
                raise RuntimeError(
                    "PD disagg: PP>1 not supported for MiniMax sparse index yet."
                )
            if peer_info.decode_tp_size != self.attn_tp_size:
                raise RuntimeError(
                    "PD disagg: heterogeneous TP not supported for MiniMax sparse index yet."
                )

        common_len = min(src_state_indices.size, dst_state_indices.size)
        if (
            state_type == "c128_state"
            and common_len == 0
            and src_state_indices.size == 0

View on GitHub (pinned to 0132848349)

Solutions

1. Set the same TP size on prefill and decode instances for SWA (non-MLA) models
2. Switch to an MLA-based model if heterogeneous TP is a hard requirement
3. Track SGLang releases for non-MLA SWA TP-slicing support

Example fix

# before
--disaggregation-prefill --tp 8   /   --disaggregation-decode --tp 4  (SWA model)

# after
--disaggregation-prefill --tp 8   /   --disaggregation-decode --tp 8

if state_type == "swa" and not conn.is_mla_backend:
    assert peer_info.decode_tp_size == conn.attn_tp_size, (
        "non-MLA SWA state transfer requires matching TP sizes"
    )

When it happens

Trigger: send_state with state_type=='swa', a non-MLA attention backend, and peer_info.decode_tp_size != self.attn_tp_size — i.e. heterogeneous TP deployment of a sliding-window model.

Common situations: Deploying a SWA/hybrid model with more prefill GPUs than decode GPUs (common cost optimization), which works for MLA models but is unimplemented for standard GQA/SWA KV layouts.

Related errors

• Slice size exceeds destination token capacity for TP slice t
• PD disagg: heterogeneous TP not supported for MiniMax sparse
• PD decode DCP requires an MLA or hybrid-MLA KV pool.
• SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models
• Head slice size evaluates to zero

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/026e7eb5aec78dea. Report an issue: GitHub.