{"record":{"id":"213d04cf659a6918","repo":"sgl-project/sglang","slug":"pd-disaggregation-does-not-support-pd-different-tp","errorCode":null,"errorMessage":"PD Disaggregation does NOT support PD different TP sizes for non-MLA {st.upper()} hybrid models yet.","messagePattern":"PD Disaggregation does NOT support PD different TP sizes for non-MLA (.+?) hybrid models yet\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/disaggregation/mooncake/conn.py","lineNumber":1344,"sourceCode":"                            req,\n                            indices,\n                            src_data_ptrs,\n                            src_item_lens,\n                            dst_data_ptrs,\n                            dst_indices,\n                            src_state_layer_ids,\n                            dst_state_layer_ids,\n                        )\n                        or rc\n                    )\n            elif self._is_generic_kvcache_state_type(st):\n                if (\n                    target_rank_registration_info is not None\n                    and not self.is_mla_backend\n                    and self.attn_tp_size\n                    != target_rank_registration_info.dst_attn_tp_size\n                ):\n                    raise RuntimeError(\n                        f\"PD Disaggregation does NOT support PD different TP sizes for non-MLA {st.upper()} hybrid models yet.\"\n                    )\n                src_indices = list(indices)\n                dst_indices_local = list(dst_indices)\n                if (\n                    st == StateType.C128_STATE\n                    and len(src_indices) == 0\n                    and len(dst_indices_local) == 0\n                ):\n                    continue\n                if len(src_indices) != len(dst_indices_local):\n                    # These components are position- or request-indexed:\n                    # truncating silently misaligns rows and corrupts KV.\n                    # Paged SWA/DSA tolerate a 1-page drift -> keep the\n                    # lenient truncation below.\n                    if self._requires_exact_state_index_match(st):\n                        raise RuntimeError(\n                            f\"{st.upper()} state index length mismatch: \"","sourceCodeStart":1326,"sourceCodeEnd":1362,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/mooncake/conn.py#L1326-L1362","documentation":"For non-MLA hybrid models, extra state (e.g. C128_STATE, SWA) cannot be redistributed across differing attention TP decompositions, so maybe_send_extra hard-fails when target registration reports a different dst_attn_tp_size.","triggerScenarios":"PD disaggregation of a hybrid non-MLA model with prefill attn TP != decode attn TP while transferring extra states in maybe_send_extra.","commonSituations":"Heterogeneous TP deployments that work for MLA models but not for non-MLA hybrids like mamba/mamba2 hybrids.","solutions":["Use identical attention TP sizes for prefill and decode","Switch to an MLA-backend model if heterogeneous TP is required","Track upstream support for heterogeneous TP on non-MLA hybrids"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"assert is_mla_backend or prefill_attn_tp == decode_attn_tp","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Config validation at deploy time for hybrid models","Document supported heterogeneous-TP matrix per backend"],"tags":["pd-disagg","tp-degree-mismatch","hybrid-model","non-mla"],"backgroundTag":"tp-degree-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}