sgl-project/sglang · error · RuntimeError

PD state transfer failed: state component count mismatch (lo

Error message

PD state transfer failed: state component count mismatch (local={len(self.state_mem_descs)}, remote={len(peer_info.dst_state_mem_descs)})

What it means

The number of local state components (state_mem_descs) does not equal the number of remote state descriptors (peer_info.dst_state_mem_descs) exchanged during registration. State transfer requires a one-to-one mapping between prefill-side and decode-side state tensors, so a count mismatch aborts the transfer.

Source

Thrown at python/sglang/srt/disaggregation/mori/conn.py:1201

    def send_state(
        self,
        peer_info: KVArgsRegisterInfo,
        src_state_indices: List[npt.NDArray[np.int32]],
        dst_state_indices: List[npt.NDArray[np.int32]],
    ) -> List[TransferStatus]:
        # Guard: no local state tensors -> no-op (e.g. SWA layers=0 on this PP rank)
        if not self.state_mem_descs:
            return []

        state_types = self.kv_args.state_types
        if not state_types:
            raise RuntimeError(
                "PD state transfer failed: kv_args.state_types is empty but "
                "state_indices were provided"
            )

        if len(peer_info.dst_state_mem_descs) != len(self.state_mem_descs):
            raise RuntimeError(
                f"PD state transfer failed: state component count mismatch "
                f"(local={len(self.state_mem_descs)}, "
                f"remote={len(peer_info.dst_state_mem_descs)})"
            )

        src_state_item_lens = self.kv_args.state_item_lens
        src_state_dim_per_tensor = self.kv_args.state_dim_per_tensor

        statuses: List[TransferStatus] = []
        for i, st in enumerate(state_types):
            src_indices = src_state_indices[i] if i < len(src_state_indices) else None
            dst_indices = dst_state_indices[i] if i < len(dst_state_indices) else None
            if src_indices is None or src_indices.size == 0:
                continue
            if dst_indices is None or dst_indices.size == 0:
                continue

            src_descs = self.state_mem_descs[i]

View on GitHub (pinned to 0132848349)

Solutions

1. Make model, hybrid-state, and PP/TP configuration identical on prefill and decode instances
2. Upgrade/downgrade so both sides run the same SGLang release (state registration layout must match)
3. Log len(self.state_mem_descs) and the peer descriptor count to identify which component list diverges

if conn.state_mem_descs:
    assert len(peer_info.dst_state_mem_descs) == len(conn.state_mem_descs), (
        f"state count mismatch local={len(conn.state_mem_descs)} "
        f"remote={len(peer_info.dst_state_mem_descs)}"
    )

When it happens

Trigger: send_state on a PD setup where the prefill instance registers N state tensors but the decode instance registered M != N — e.g. state_types lists different components per side, or layer/PP partitioning assigns different state counts to corresponding ranks.

Common situations: Prefill and decode running different SGLang versions with changed state registration, different hybrid-model configs (one side enabling SWA/Mamba state the other lacks), or mismatched PP partitions giving ranks different state component counts.

Related errors

• PD state transfer failed: kv_args.state_types is empty but s
• PD state transfer failed: unknown state_type={st}
• PD state transfer failed: mamba requires single state index,
• NIXL memory registration failed for state tensors
• Serve backend {name!r} uses API version {backend.api_version

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b0338dd74aa6b751. Report an issue: GitHub.