sgl-project/sglang · error · RuntimeError

HiSparse destination device indices are not supported by PD

Error message

HiSparse destination device indices are not supported by PD DCP relayout

What it means

When the target rank requires DCP (DeepSeek sparse cache) relayout, the relayout derives destination indices from dst_kv_indices only; per-request destination device indices cannot be honored and the transfer is rejected.

Source

Thrown at python/sglang/srt/disaggregation/mooncake/conn.py:1676

                                self.sync_status_to_decode_endpoint(
                                    req.endpoint,
                                    req.dst_port,
                                    req.room,
                                    KVPoll.Failed,
                                    prefill_unique_rank,
                                )
                                break

                        target_rank_registration_info: KVArgsRegisterInfo = (
                            self.decode_kv_args_table[req.mooncake_session_id]
                        )
                        is_dcp_transfer = (
                            target_rank_registration_info.requires_dcp_relayout
                        )
                        chunked_dst_device_kv_indice = None
                        if is_dcp_transfer:
                            if req.dst_device_kv_indices is not None:
                                raise RuntimeError(
                                    "HiSparse destination device indices are not "
                                    "supported by PD DCP relayout"
                                )
                            chunked_dst_kv_indice = req.dst_kv_indices
                        else:
                            chunked_dst_kv_indice = req.dst_kv_indices[
                                kv_chunk.index_slice
                            ]
                            if req.dst_device_kv_indices is not None:
                                chunked_dst_device_kv_indice = (
                                    req.dst_device_kv_indices[kv_chunk.index_slice]
                                )

                            # NOTE: This is temporarily a workaround to deal with the case where the prefill_kv_indices
                            # is mismatched with the dst_kv_indices when page size > 1, this should never happen.
                            if len(chunked_dst_kv_indice) < len(
                                kv_chunk.prefill_kv_indices
                            ):

View on GitHub (pinned to 0132848349)

Solutions

1. Disable the feature producing dst_device_kv_indices (e.g. hicache device placement for these requests)
2. Disable DCP relayout on the decode side so the normal path runs
3. Route such requests to a non-DCP decode pool

if target_registration.requires_dcp_relayout:
    req.dst_device_kv_indices = None  # route via dst_kv_indices only

When it happens

Trigger: transfer_worker handling a request with dst_device_kv_indices != None where target_rank_registration_info.requires_dcp_relayout is true.

Common situations: Mixing HiCache device-resident requests with a decode side that performs DCP relayout — an unsupported combination.

Related errors

• Page size mismatch: prefill server has page_size={info.page_
• get_split_heads_page_buffer_meta requires layout='page_head'
• Unsupported layout for models with head_dim != v_head_dim: {
• /v1/models ${response.status}
• Unsupported msgpack byte ${b}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b1eae779957fbb8b. Report an issue: GitHub.