sgl-project/sglang · error · RuntimeError

PD disagg: heterogeneous TP not supported for MiniMax sparse

Error message

PD disagg: heterogeneous TP not supported for MiniMax sparse index yet.

What it means

MiniMax sparse index (minimax_index_k) state transfer requires prefill and decode to use the same TP size; if peer_info.decode_tp_size differs from the local attention TP size the transfer aborts. The sparse index layout is TP-sharded and has no re-sharding logic in the transfer path.

Source

Thrown at python/sglang/srt/disaggregation/mori/conn.py:1383

        state_type: str,
    ) -> List[TransferStatus]:
        # TP mismatch check for non-MLA SWA
        if (
            state_type == "swa"
            and not self.is_mla_backend
            and peer_info.decode_tp_size != self.attn_tp_size
        ):
            raise RuntimeError(
                f"PD state transfer does not support TP-mismatched non-MLA SWA models "
                f"(prefill_tp_size={self.attn_tp_size}, decode_tp_size={peer_info.decode_tp_size})"
            )
        if state_type == "minimax_index_k":
            if self.pp_size is not None and self.pp_size > 1:
                raise RuntimeError(
                    "PD disagg: PP>1 not supported for MiniMax sparse index yet."
                )
            if peer_info.decode_tp_size != self.attn_tp_size:
                raise RuntimeError(
                    "PD disagg: heterogeneous TP not supported for MiniMax sparse index yet."
                )

        common_len = min(src_state_indices.size, dst_state_indices.size)
        if (
            state_type == "c128_state"
            and common_len == 0
            and src_state_indices.size == 0
            and dst_state_indices.size == 0
        ):
            return []
        if common_len == 0 and max(src_state_indices.size, dst_state_indices.size) > 0:
            raise RuntimeError(
                f"No overlapping state indices for state_type={state_type}"
            )
        if src_state_indices.size != dst_state_indices.size:
            # These components are position- or request-indexed: truncating
            # silently misaligns rows and corrupts KV. Paged swa/dsa tolerate

View on GitHub (pinned to 0132848349)

Solutions

1. Launch prefill and decode with identical TP sizes for MiniMax models
2. Re-plan capacity using equal TP (scale ranks per node, not TP degree) until heterogeneous TP support lands
3. Verify with --tp printed on both server logs that the degrees actually match

Example fix

# before
--disaggregation-prefill --tp 8   /   --disaggregation-decode --tp 4

# after
--disaggregation-prefill --tp 8   /   --disaggregation-decode --tp 8

if "minimax_index_k" in (kv_args.state_types or []):
    assert peer_info.decode_tp_size == conn.attn_tp_size, (
        "MiniMax sparse index requires identical TP on prefill and decode"
    )

When it happens

Trigger: send_state with state_type=='minimax_index_k' where the decode instance was launched with a different --tp than the prefill instance (e.g. prefill tp=8, decode tp=4).

Common situations: Heterogeneous PD deployments sized independently for prefill and decode throughput; supported for some KV layouts but not for MiniMax sparse index state.

Related errors

• Slice size exceeds destination token capacity for TP slice t
• PD state transfer does not support TP-mismatched non-MLA SWA
• PD disagg: PP>1 not supported for MiniMax sparse index yet.
• PD disaggregation for MiniMax sparse layers with index value
• sparse_mla_q8kv8_prefill_fwd requires h_q padded to a positi

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/278e5ae2169e375c. Report an issue: GitHub.