{"record":{"id":"2d34f5162397b7e8","repo":"sgl-project/sglang","slug":"enable-unified-memory-does-not-support-different","errorCode":null,"errorMessage":"--enable-unified-memory does not support different prefill / decode attention TP sizes (prefill={self.attn_tp_size}, decode={dst_attn_tp_size}): the whole-envelope state cannot be TP-resliced.","messagePattern":"--enable-unified-memory does not support different prefill / decode attention TP sizes \\(prefill=(.+?), decode=(.+?)\\): the whole-envelope state cannot be TP-resliced\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/disaggregation/mooncake/conn.py","lineNumber":789,"sourceCode":"\n        The unified memory pool registers ONE whole-envelope region and\n        addresses the destination as ``dst_ptr + page_id * item_len`` using OUR\n        ``item_len``, so a peer on a different page size / spec, or without\n        unified memory, would take envelope-sized blocks at the wrong offsets.\n        Must run before the first RDMA write.\n\n        Scoped to unified memory by config, not by region count: a non-unified\n        PP stage owning a single full-attention layer also registers one region,\n        and `_send_kvcache_generic` pairs that with the peer by layer id.\n        \"\"\"\n        if not get_memory().enable_unified_memory:\n            return\n        if dst_attn_tp_size is not None and self.attn_tp_size != dst_attn_tp_size:\n            # The unified mamba state ships as one whole-slot envelope with no\n            # per-tensor dims, so `_send_mamba_state_slice` cannot reslice it and\n            # silently falls back to an unsliced copy. Reject here, before any KV\n            # is written, rather than in `maybe_send_extra` afterwards.\n            raise RuntimeError(\n                \"--enable-unified-memory does not support different prefill / \"\n                f\"decode attention TP sizes (prefill={self.attn_tp_size}, \"\n                f\"decode={dst_attn_tp_size}): the whole-envelope state cannot \"\n                \"be TP-resliced.\"\n            )\n        src_item_lens = self.kv_args.kv_item_lens\n        if (\n            len(src_item_lens) != 1\n            or len(dst_kv_ptrs) != 1\n            or dst_kv_item_len is None\n            or src_item_lens[0] != dst_kv_item_len\n        ):\n            raise RuntimeError(\n                \"PD KV layout mismatch on the whole-envelope path: prefill has \"\n                f\"{len(src_item_lens)} KV region(s) with item_lens=\"\n                f\"{src_item_lens}, decode has {len(dst_kv_ptrs)} with item_len=\"\n                f\"{dst_kv_item_len}. With --enable-unified-memory both sides \"\n                \"must enable it and use the same page size and model spec.\"","sourceCodeStart":771,"sourceCodeEnd":807,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/mooncake/conn.py#L771-L807","documentation":"With --enable-unified-memory the mamba/whole-envelope state ships as one slot-sized blob with no per-tensor dims, so it cannot be resliced across a different attention TP decomposition. _validate_envelope_kv_layout rejects differing prefill vs decode attn TP sizes before any KV is written.","triggerScenarios":"Starting PD disaggregation with --enable-unified-memory where prefill attention-TP != decode attention-TP on a hybrid (mamba) model.","commonSituations":"Operators trying heterogeneous TP (e.g. prefill tp=8, decode tp=4) to right-size phases; unsupported for unified-memory hybrid models.","solutions":["Set the same attention TP size on both prefill and decode instances","Drop --enable-unified-memory if heterogeneous TP is a hard requirement","Use an MLA-backend model, which supports TP reslicing"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"assert prefill_attn_tp == decode_attn_tp or not enable_unified_memory, \"unified-memory requires equal attn TP\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate TP topology in the launch script before starting PD pair","Keep a topology manifest shared by prefill and decode"],"tags":["pd-disagg","unified-memory","tensor-parallel","mamba"],"backgroundTag":"tp-degree-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}