sgl-project/sglang · critical · ValueError

Destination MLA KV descriptors do not match prefill pp confi

Error message

Destination MLA KV descriptors do not match prefill pp configuration

What it means

Raised when sending an MLA KV cache in a Mori prefill-decode disaggregation setup: the local prefill rank holds num_local_layers MLA KV descriptors, and prefill_start_layer + num_local_layers exceeds the number of descriptors registered by the destination decode instance. This means the layer-range arithmetic derived from the prefill PP (pipeline parallelism) configuration does not fit inside the decode side's descriptor table, i.e. the two processes disagree about layer counts or layer offsets.

Source

Thrown at python/sglang/srt/disaggregation/mori/conn.py:878

        if len(dst_mem_descs) < 2 or end_layer > dst_total_layers:
            raise ValueError(
                "Destination KV descriptors do not match prefill pp configuration"
            )
        dst_k_descs = dst_mem_descs[start_layer:end_layer]
        dst_v_descs = dst_mem_descs[
            dst_total_layers + start_layer : dst_total_layers + end_layer
        ]
        return src_k_descs, src_v_descs, dst_k_descs, dst_v_descs, num_local_layers

    def _get_mla_mem_desc_slices(
        self, dst_mem_descs: List[MemoryDesc]
    ) -> tuple[List[MemoryDesc], List[MemoryDesc], int]:
        src_descs = self.kv_mem_descs
        num_local_layers = len(src_descs)
        start_layer = self.kv_args.prefill_start_layer
        end_layer = start_layer + num_local_layers
        if end_layer > len(dst_mem_descs):
            raise ValueError(
                "Destination MLA KV descriptors do not match prefill pp configuration"
            )
        dst_slice = dst_mem_descs[start_layer:end_layer]
        return src_descs, dst_slice, num_local_layers

    def _submit_batch_transfer_plan(
        self,
        src_desc: MemoryDesc,
        dst_desc: MemoryDesc,
        plan: BatchTransferPlan,
    ) -> List[TransferStatus]:
        if plan.empty():
            return []

        transfer_uid = self.engine.allocate_transfer_uid()

        statuses = self.engine.batch_write(
            [src_desc],

View on GitHub (pinned to 0132848349)

Solutions

Verify prefill and decode servers use identical --pp-size and the same model (same number of layers)
Check that kv_args.prefill_start_layer for this PP rank plus its local layer count stays within the decode instance's total descriptor count
Re-register the KV descriptors (restart both engines) so the exchanged peer metadata matches current topology
If using custom layer partitioning, validate prefill_start_layer/prefill_end_layer config values against the model's layer count

Example fix

# before
python -m sglang.launch_server --model ... --disaggregation-prefill --pp-size 2  # prefill
python -m sglang.launch_server --model ... --disaggregation-decode --pp-size 1  # decode

# after: match PP topology on both sides
python -m sglang.launch_server --model ... --disaggregation-prefill --pp-size 2
python -m sglang.launch_server --model ... --disaggregation-decode --pp-size 2

Defensive patterns

Strategy: validation

Validate before calling

start = kv_args.prefill_start_layer
local = len(conn.kv_mem_descs)
assert start + local <= len(peer_info.dst_mem_descs), (
    f"layer range [{start}, {start + local}) exceeds decode descriptors "
    f"({len(peer_info.dst_mem_descs)}) — check PP config parity"
)

Prevention

Always launch prefill and decode with the same --pp-size and model
Validate exchanged descriptor metadata at registration time rather than at transfer time
Add a startup assertion comparing layer counts across PD peers

When it happens

Trigger: Calling send_kvcache (via _submit_kv_transfer) with kv_args.prefill_start_layer set such that prefill_start_layer + len(kv_mem_descs) > len(dst_mem_descs). Happens when PP sizes, layer partitioning, or model configuration differ between the prefill and decode engines.

Common situations: Launching prefill and decode instances with different --pp-size values, different model versions/layer counts, mismatched pipeline layer partition configs, or a stale peer registration (descriptor metadata exchanged at bootstrap does not match the actual engine topology).

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1cf1ea870b01f5b5. Report an issue: GitHub.