{"record":{"id":"b0338dd74aa6b751","repo":"sgl-project/sglang","slug":"pd-state-transfer-failed-state-component-count-mi","errorCode":null,"errorMessage":"PD state transfer failed: state component count mismatch (local={len(self.state_mem_descs)}, remote={len(peer_info.dst_state_mem_descs)})","messagePattern":"PD state transfer failed: state component count mismatch \\(local=(.+?), remote=(.+?)\\)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/disaggregation/mori/conn.py","lineNumber":1201,"sourceCode":"    def send_state(\n        self,\n        peer_info: KVArgsRegisterInfo,\n        src_state_indices: List[npt.NDArray[np.int32]],\n        dst_state_indices: List[npt.NDArray[np.int32]],\n    ) -> List[TransferStatus]:\n        # Guard: no local state tensors -> no-op (e.g. SWA layers=0 on this PP rank)\n        if not self.state_mem_descs:\n            return []\n\n        state_types = self.kv_args.state_types\n        if not state_types:\n            raise RuntimeError(\n                \"PD state transfer failed: kv_args.state_types is empty but \"\n                \"state_indices were provided\"\n            )\n\n        if len(peer_info.dst_state_mem_descs) != len(self.state_mem_descs):\n            raise RuntimeError(\n                f\"PD state transfer failed: state component count mismatch \"\n                f\"(local={len(self.state_mem_descs)}, \"\n                f\"remote={len(peer_info.dst_state_mem_descs)})\"\n            )\n\n        src_state_item_lens = self.kv_args.state_item_lens\n        src_state_dim_per_tensor = self.kv_args.state_dim_per_tensor\n\n        statuses: List[TransferStatus] = []\n        for i, st in enumerate(state_types):\n            src_indices = src_state_indices[i] if i < len(src_state_indices) else None\n            dst_indices = dst_state_indices[i] if i < len(dst_state_indices) else None\n            if src_indices is None or src_indices.size == 0:\n                continue\n            if dst_indices is None or dst_indices.size == 0:\n                continue\n\n            src_descs = self.state_mem_descs[i]","sourceCodeStart":1183,"sourceCodeEnd":1219,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/mori/conn.py#L1183-L1219","documentation":"The number of local state components (state_mem_descs) does not equal the number of remote state descriptors (peer_info.dst_state_mem_descs) exchanged during registration. State transfer requires a one-to-one mapping between prefill-side and decode-side state tensors, so a count mismatch aborts the transfer.","triggerScenarios":"send_state on a PD setup where the prefill instance registers N state tensors but the decode instance registered M != N — e.g. state_types lists different components per side, or layer/PP partitioning assigns different state counts to corresponding ranks.","commonSituations":"Prefill and decode running different SGLang versions with changed state registration, different hybrid-model configs (one side enabling SWA/Mamba state the other lacks), or mismatched PP partitions giving ranks different state component counts.","solutions":["Make model, hybrid-state, and PP/TP configuration identical on prefill and decode instances","Upgrade/downgrade so both sides run the same SGLang release (state registration layout must match)","Log len(self.state_mem_descs) and the peer descriptor count to identify which component list diverges"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if conn.state_mem_descs:\n    assert len(peer_info.dst_state_mem_descs) == len(conn.state_mem_descs), (\n        f\"state count mismatch local={len(conn.state_mem_descs)} \"\n        f\"remote={len(peer_info.dst_state_mem_descs)}\"\n    )\n","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep model and hybrid-state config identical on both PD sides","Version-pin SGLang identically on prefill and decode hosts","Compare registration metadata after peer exchange"],"tags":["disaggregation","hybrid-model","state-transfer","version-mismatch"],"backgroundTag":"count-mismatch-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}