sgl-project/sglang · error · ValueError

Destination KV descriptors do not match prefill pp configura

Error message

Destination KV descriptors do not match prefill pp configuration

What it means

When slicing destination descriptors for MHA transfer, the prefill's [start_layer, start_layer+num_local_layers) window must fit within the destination's total layer count; otherwise the PP configuration is inconsistent and ValueError is raised.

Source

Thrown at python/sglang/srt/disaggregation/mori/conn.py:861

    def _get_mha_mem_desc_slices(
        self, dst_mem_descs: List[MemoryDesc]
    ) -> tuple[
        List[MemoryDesc], List[MemoryDesc], List[MemoryDesc], List[MemoryDesc], int
    ]:
        src_descs = self.kv_mem_descs
        if not src_descs:
            raise RuntimeError("KV memory descriptors are empty on prefill side")

        num_local_layers = len(src_descs) // 2
        src_k_descs = src_descs[:num_local_layers]
        src_v_descs = src_descs[num_local_layers:]

        start_layer = self.kv_args.prefill_start_layer
        end_layer = start_layer + num_local_layers
        dst_total_layers = len(dst_mem_descs) // 2
        if len(dst_mem_descs) < 2 or end_layer > dst_total_layers:
            raise ValueError(
                "Destination KV descriptors do not match prefill pp configuration"
            )
        dst_k_descs = dst_mem_descs[start_layer:end_layer]
        dst_v_descs = dst_mem_descs[
            dst_total_layers + start_layer : dst_total_layers + end_layer
        ]
        return src_k_descs, src_v_descs, dst_k_descs, dst_v_descs, num_local_layers

    def _get_mla_mem_desc_slices(
        self, dst_mem_descs: List[MemoryDesc]
    ) -> tuple[List[MemoryDesc], List[MemoryDesc], int]:
        src_descs = self.kv_mem_descs
        num_local_layers = len(src_descs)
        start_layer = self.kv_args.prefill_start_layer
        end_layer = start_layer + num_local_layers
        if end_layer > len(dst_mem_descs):
            raise ValueError(
                "Destination MLA KV descriptors do not match prefill pp configuration"

View on GitHub (pinned to 0132848349)

Solutions

1. Match pipeline-parallel layer ranges between prefill and decode (same pp size and layer counts)
2. Confirm both sides load the same model checkpoint
3. Check prefill_start_layer in kv_args matches the deployment topology

dst_total = len(dst_mem_descs) // 2
assert dst_total >= prefill_start_layer + num_local_layers, (dst_total, prefill_start_layer, num_local_layers)

catch ValueError at send time; fail fast with a clear PD topology mismatch message

When it happens

Trigger: len(dst_mem_descs)//2 < end_layer or fewer than 2 dst descriptors — decode side has fewer KV layers than the prefill's slice requires.

Common situations: Prefill and decode launched with different --pp-size or layer partitioning, or a decode model with fewer layers (wrong checkpoint).

Related errors

• KV memory descriptors are empty on prefill side
• Destination MLA KV descriptors do not match prefill pp confi
• v_cache must be provided
• k_cache can only be None when only_qv=True
• rope_pool_fused expects pool tensors to be 3-D

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d94cad9cab09972a. Report an issue: GitHub.