sgl-project/sglang · critical · RuntimeError

KV memory descriptors are empty on prefill side

Error message

KV memory descriptors are empty on prefill side

What it means

The Mori MHA send path requires the prefill-side kv_mem_descs to be populated before _get_mha_mem_desc_slices can compute layer slices; an empty list means descriptor registration never happened on this instance.

Source

Thrown at python/sglang/srt/disaggregation/mori/conn.py:851

            logger.debug("Remote peer %s already registered. Skipping.", engine_key)
            return
        self.engine.register_remote_engine(register_info.engine_desc)
        self.decode_kv_args_table[engine_key] = register_info
        logger.debug(
            "Registered decode peer %s (%s:%s)",
            engine_key,
            register_info.endpoint,
            register_info.dst_port,
        )

    def _get_mha_mem_desc_slices(
        self, dst_mem_descs: List[MemoryDesc]
    ) -> tuple[
        List[MemoryDesc], List[MemoryDesc], List[MemoryDesc], List[MemoryDesc], int
    ]:
        src_descs = self.kv_mem_descs
        if not src_descs:
            raise RuntimeError("KV memory descriptors are empty on prefill side")

        num_local_layers = len(src_descs) // 2
        src_k_descs = src_descs[:num_local_layers]
        src_v_descs = src_descs[num_local_layers:]

        start_layer = self.kv_args.prefill_start_layer
        end_layer = start_layer + num_local_layers
        dst_total_layers = len(dst_mem_descs) // 2
        if len(dst_mem_descs) < 2 or end_layer > dst_total_layers:
            raise ValueError(
                "Destination KV descriptors do not match prefill pp configuration"
            )
        dst_k_descs = dst_mem_descs[start_layer:end_layer]
        dst_v_descs = dst_mem_descs[
            dst_total_layers + start_layer : dst_total_layers + end_layer
        ]
        return src_k_descs, src_v_descs, dst_k_descs, dst_v_descs, num_local_layers

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure mori endpoint init/registration completes before any send_kvcache call (check startup logs for descriptor publication)
2. Restart the prefill instance if registration was skipped
3. Verify allocator actually created KV pools (non-empty model load)

assert kv_mem_descs, "mori descriptors not registered yet; defer send_kvcache"

def descriptors_ready(conn) -> bool: return len(conn.kv_mem_descs) > 0

catch RuntimeError 'KV memory descriptors are empty'; queue the send until descriptor registration completes rather than crashing

When it happens

Trigger: send_kvcache invoked before local memory descriptors were registered/published (init ordering bug), or descriptor registration failed silently.

Common situations: Race between scheduler issuing sends and the mori endpoint finishing descriptor exchange; failed startup registration swallowed by retries.

Related errors

• Destination KV descriptors do not match prefill pp configura
• v_cache must be provided
• k_cache can only be None when only_qv=True
• rope_pool_fused expects pool tensors to be 3-D
• k_pool has incompatible shape {k_pool.shape}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/795554d942550d42. Report an issue: GitHub.