sgl-project/sglang · error · RuntimeError

PD state transfer failed: kv_args.state_types is empty but s

Error message

PD state transfer failed: kv_args.state_types is empty but state_indices were provided

What it means

send_state was invoked with state_indices to transfer (e.g. Mamba/SWA recurrent state for hybrid models), but kv_args.state_types is empty on this instance. The library refuses to transfer state whose type taxonomy was never declared, since it cannot map indices to state components without it.

Source

Thrown at python/sglang/srt/disaggregation/mori/conn.py:1195

                str(aux_index).encode("ascii"),
                struct.pack(">I", len(data)),
                data,
            ]
        )

    def send_state(
        self,
        peer_info: KVArgsRegisterInfo,
        src_state_indices: List[npt.NDArray[np.int32]],
        dst_state_indices: List[npt.NDArray[np.int32]],
    ) -> List[TransferStatus]:
        # Guard: no local state tensors -> no-op (e.g. SWA layers=0 on this PP rank)
        if not self.state_mem_descs:
            return []

        state_types = self.kv_args.state_types
        if not state_types:
            raise RuntimeError(
                "PD state transfer failed: kv_args.state_types is empty but "
                "state_indices were provided"
            )

        if len(peer_info.dst_state_mem_descs) != len(self.state_mem_descs):
            raise RuntimeError(
                f"PD state transfer failed: state component count mismatch "
                f"(local={len(self.state_mem_descs)}, "
                f"remote={len(peer_info.dst_state_mem_descs)})"
            )

        src_state_item_lens = self.kv_args.state_item_lens
        src_state_dim_per_tensor = self.kv_args.state_dim_per_tensor

        statuses: List[TransferStatus] = []
        for i, st in enumerate(state_types):
            src_indices = src_state_indices[i] if i < len(src_state_indices) else None
            dst_indices = dst_state_indices[i] if i < len(dst_state_indices) else None

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure the attention backend/model integration populates kv_args.state_types alongside state memory descriptors (update SGLang or your backend adapter)
2. Check that the hybrid state configuration (e.g. --mamba-full-memory-ratio, SWA settings) that enables state tracking also declares its types
3. Verify both prefill and decode run the same SGLang version so state_types is symmetric

if state_indices is not None and state_indices.numel() > 0:
    assert kv_args.state_types, (
        "state_indices provided but kv_args.state_types is empty"
    )

When it happens

Trigger: Calling send_kvcache/_submit_kv_transfer for a hybrid model where state_mem_descs is non-empty (state pool allocated) but the engine never populated kv_args.state_types — e.g. a version/config path that registers state buffers without declaring their types.

Common situations: Upgrading SGLang where state_types was added to KVArgs but the custom attention backend or model integration was not updated to fill it; enabling a hybrid (Mamba/GDN/SWA) model with an attention backend that skips state type registration.

Related errors

• PD state transfer failed: state component count mismatch (lo
• PD state transfer failed: mamba requires single state index,
• mooncake encoder_transfer_backend requires HTTP encoders; us
• PD state transfer failed: unknown state_type={st}
• NIXL memory registration failed for state tensors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d12fb36ee5c933e7. Report an issue: GitHub.