sgl-project/sglang · error · RuntimeError

PD Disaggregation does NOT support PD different TP sizes for

Error message

PD Disaggregation does NOT support PD different TP sizes for non-MLA {st.upper()} hybrid models yet.

What it means

For non-MLA hybrid models, extra state (e.g. C128_STATE, SWA) cannot be redistributed across differing attention TP decompositions, so maybe_send_extra hard-fails when target registration reports a different dst_attn_tp_size.

Source

Thrown at python/sglang/srt/disaggregation/mooncake/conn.py:1344

                            req,
                            indices,
                            src_data_ptrs,
                            src_item_lens,
                            dst_data_ptrs,
                            dst_indices,
                            src_state_layer_ids,
                            dst_state_layer_ids,
                        )
                        or rc
                    )
            elif self._is_generic_kvcache_state_type(st):
                if (
                    target_rank_registration_info is not None
                    and not self.is_mla_backend
                    and self.attn_tp_size
                    != target_rank_registration_info.dst_attn_tp_size
                ):
                    raise RuntimeError(
                        f"PD Disaggregation does NOT support PD different TP sizes for non-MLA {st.upper()} hybrid models yet."
                    )
                src_indices = list(indices)
                dst_indices_local = list(dst_indices)
                if (
                    st == StateType.C128_STATE
                    and len(src_indices) == 0
                    and len(dst_indices_local) == 0
                ):
                    continue
                if len(src_indices) != len(dst_indices_local):
                    # These components are position- or request-indexed:
                    # truncating silently misaligns rows and corrupts KV.
                    # Paged SWA/DSA tolerate a 1-page drift -> keep the
                    # lenient truncation below.
                    if self._requires_exact_state_index_match(st):
                        raise RuntimeError(
                            f"{st.upper()} state index length mismatch: "

View on GitHub (pinned to 0132848349)

Solutions

1. Use identical attention TP sizes for prefill and decode
2. Switch to an MLA-backend model if heterogeneous TP is required
3. Track upstream support for heterogeneous TP on non-MLA hybrids

assert is_mla_backend or prefill_attn_tp == decode_attn_tp

When it happens

Trigger: PD disaggregation of a hybrid non-MLA model with prefill attn TP != decode attn TP while transferring extra states in maybe_send_extra.

Common situations: Heterogeneous TP deployments that work for MLA models but not for non-MLA hybrids like mamba/mamba2 hybrids.

Related errors

• Mamba state layouts differ between prefill and decode (src i
• PD disagg: heterogeneous TP not supported for MiniMax sparse
• Page size mismatch: prefill server has page_size={info.page_
• KV cache dtype mismatch: prefill server has kv_cache_dtype={
• --enable-unified-memory does not support different prefill /

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/213d04cf659a6918. Report an issue: GitHub.