{"record":{"id":"2e31a27b0b9d2dac","repo":"sgl-project/sglang","slug":"pd-kv-layout-mismatch-on-the-whole-envelope-path","errorCode":null,"errorMessage":"PD KV layout mismatch on the whole-envelope path: prefill has {len(src_item_lens)} KV region(s) with item_lens={src_item_lens}, decode has {len(dst_kv_ptrs)} with item_len={dst_kv_item_len}. With --enable-unified-memory both sides must enable it and use the same page size and model spec.","messagePattern":"PD KV layout mismatch on the whole-envelope path: prefill has (.+?) KV region\\(s\\) with item_lens=(.+?), decode has (.+?) with item_len=(.+?)\\. With --enable-unified-memory both sides must enable it and use the same page size and model spec\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/disaggregation/mooncake/conn.py","lineNumber":802,"sourceCode":"        if dst_attn_tp_size is not None and self.attn_tp_size != dst_attn_tp_size:\n            # The unified mamba state ships as one whole-slot envelope with no\n            # per-tensor dims, so `_send_mamba_state_slice` cannot reslice it and\n            # silently falls back to an unsliced copy. Reject here, before any KV\n            # is written, rather than in `maybe_send_extra` afterwards.\n            raise RuntimeError(\n                \"--enable-unified-memory does not support different prefill / \"\n                f\"decode attention TP sizes (prefill={self.attn_tp_size}, \"\n                f\"decode={dst_attn_tp_size}): the whole-envelope state cannot \"\n                \"be TP-resliced.\"\n            )\n        src_item_lens = self.kv_args.kv_item_lens\n        if (\n            len(src_item_lens) != 1\n            or len(dst_kv_ptrs) != 1\n            or dst_kv_item_len is None\n            or src_item_lens[0] != dst_kv_item_len\n        ):\n            raise RuntimeError(\n                \"PD KV layout mismatch on the whole-envelope path: prefill has \"\n                f\"{len(src_item_lens)} KV region(s) with item_lens=\"\n                f\"{src_item_lens}, decode has {len(dst_kv_ptrs)} with item_len=\"\n                f\"{dst_kv_item_len}. With --enable-unified-memory both sides \"\n                \"must enable it and use the same page size and model spec.\"\n            )\n\n    def _await_transfer_futures(self, futures) -> int:\n        \"\"\"Await a chunk's per-layer RDMA writes; return the first non-zero status.\n        cancel() is a no-op for a running future, so with deferred release on we\n        still drain the running ones before returning (no write may outlive this\n        call, which the drain-ack relies on). Off: original early-return.\"\"\"\n        ret = 0\n        for future in concurrent.futures.as_completed(futures):\n            try:\n                status = future.result()\n            except concurrent.futures.CancelledError:\n                continue","sourceCodeStart":784,"sourceCodeEnd":820,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/mooncake/conn.py#L784-L820","documentation":"On the whole-envelope path (unified memory), the layout validation requires exactly one KV region on each side with identical item_len. Any deviation — multiple regions, missing dst item_len, or size mismatch — means the two sides disagree on page size or model spec and the transfer is rejected.","triggerScenarios":"Prefill and decode disagreeing on page_size / model / unified-memory flag, so kv_item_lens and destination descriptors don't line up one-to-one.","commonSituations":"One side launched without --enable-unified-memory, different --page-size flags, or different model revisions between PD instances.","solutions":["Enable --enable-unified-memory on BOTH prefill and decode","Use identical --page-size on both sides","Confirm both sides load the same model spec (same checkpoint/config)","Re-check custom weight loading that alters kv_item_lens"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"assert page_size_prefill == page_size_decode\nassert unified_memory_prefill == unified_memory_decode","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Templatize PD launch configs so page-size/model flags cannot drift","Smoke-test one request after PD bring-up"],"tags":["pd-disagg","unified-memory","kv-layout","page-size"],"backgroundTag":"layout-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}