sgl-project/sglang · error · RuntimeError

PD state transfer does not support TP-mismatched non-MLA…

Error message

PD state transfer does not support TP-mismatched non-MLA SWA models (prefill_tp_size={self.attn_tp_size}, decode_tp_size={peer_info.decode_tp_size})

What it means

For SWA (sliding-window attention) state transfer on non-MLA backends, the sender requires prefill TP size to equal decode TP size. The SWA state layout depends on TP head sharding, and the transfer path has no slicing support for it unless MLA is used.

Solutions

Set the same TP size on prefill and decode instances for SWA (non-MLA) models
Switch to an MLA-based model if heterogeneous TP is a hard requirement
Track SGLang releases for non-MLA SWA TP-slicing support

Example fix

# before
--disaggregation-prefill --tp 8   /   --disaggregation-decode --tp 4  (SWA model)

# after
--disaggregation-prefill --tp 8   /   --disaggregation-decode --tp 8

Defensive patterns

Strategy: validation

Validate before calling

if state_type == "swa" and not conn.is_mla_backend:
    assert peer_info.decode_tp_size == conn.attn_tp_size, (
        "non-MLA SWA state transfer requires matching TP sizes"
    )

Prevention

Match TP sizes for non-MLA SWA models in PD deployments
Prefer MLA models when heterogeneous TP is required

When it happens

Trigger: send_state with state_type=='swa', a non-MLA attention backend, and peer_info.decode_tp_size != self.attn_tp_size — i.e. heterogeneous TP deployment of a sliding-window model.

Common situations: Deploying a SWA/hybrid model with more prefill GPUs than decode GPUs (common cost optimization), which works for MLA models but is unimplemented for standard GQA/SWA KV layouts.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/026e7eb5aec78dea. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/disaggregation/mori/conn.py:1373

        return statuses

    def _send_swa_dsa_state(
        self,
        peer_info: KVArgsRegisterInfo,
        src_state_indices: npt.NDArray[np.int32],
        dst_state_indices: npt.NDArray[np.int32],
        src_state_mem_descs: List[MemoryDesc],
        src_state_item_lens: List[int],
        dst_state_mem_descs: List[MemoryDesc],
        state_type: str,
    ) -> List[TransferStatus]:
        # TP mismatch check for non-MLA SWA
        if (
            state_type == "swa"
            and not self.is_mla_backend
            and peer_info.decode_tp_size != self.attn_tp_size
        ):
            raise RuntimeError(
                f"PD state transfer does not support TP-mismatched non-MLA SWA models "
                f"(prefill_tp_size={self.attn_tp_size}, decode_tp_size={peer_info.decode_tp_size})"
            )
        if state_type == "minimax_index_k":
            if self.pp_size is not None and self.pp_size > 1:
                raise RuntimeError(
                    "PD disagg: PP>1 not supported for MiniMax sparse index yet."
                )
            if peer_info.decode_tp_size != self.attn_tp_size:
                raise RuntimeError(
                    "PD disagg: heterogeneous TP not supported for MiniMax sparse index yet."
                )

        common_len = min(src_state_indices.size, dst_state_indices.size)
        if (
            state_type == "c128_state"
            and common_len == 0
            and src_state_indices.size == 0

View on GitHub (pinned to 0132848349)