{"record":{"id":"795554d942550d42","repo":"sgl-project/sglang","slug":"kv-memory-descriptors-are-empty-on-prefill-side","errorCode":null,"errorMessage":"KV memory descriptors are empty on prefill side","messagePattern":"KV memory descriptors are empty on prefill side","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/disaggregation/mori/conn.py","lineNumber":851,"sourceCode":"            logger.debug(\"Remote peer %s already registered. Skipping.\", engine_key)\n            return\n        self.engine.register_remote_engine(register_info.engine_desc)\n        self.decode_kv_args_table[engine_key] = register_info\n        logger.debug(\n            \"Registered decode peer %s (%s:%s)\",\n            engine_key,\n            register_info.endpoint,\n            register_info.dst_port,\n        )\n\n    def _get_mha_mem_desc_slices(\n        self, dst_mem_descs: List[MemoryDesc]\n    ) -> tuple[\n        List[MemoryDesc], List[MemoryDesc], List[MemoryDesc], List[MemoryDesc], int\n    ]:\n        src_descs = self.kv_mem_descs\n        if not src_descs:\n            raise RuntimeError(\"KV memory descriptors are empty on prefill side\")\n\n        num_local_layers = len(src_descs) // 2\n        src_k_descs = src_descs[:num_local_layers]\n        src_v_descs = src_descs[num_local_layers:]\n\n        start_layer = self.kv_args.prefill_start_layer\n        end_layer = start_layer + num_local_layers\n        dst_total_layers = len(dst_mem_descs) // 2\n        if len(dst_mem_descs) < 2 or end_layer > dst_total_layers:\n            raise ValueError(\n                \"Destination KV descriptors do not match prefill pp configuration\"\n            )\n        dst_k_descs = dst_mem_descs[start_layer:end_layer]\n        dst_v_descs = dst_mem_descs[\n            dst_total_layers + start_layer : dst_total_layers + end_layer\n        ]\n        return src_k_descs, src_v_descs, dst_k_descs, dst_v_descs, num_local_layers\n","sourceCodeStart":833,"sourceCodeEnd":869,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/mori/conn.py#L833-L869","documentation":"The Mori MHA send path requires the prefill-side kv_mem_descs to be populated before _get_mha_mem_desc_slices can compute layer slices; an empty list means descriptor registration never happened on this instance.","triggerScenarios":"send_kvcache invoked before local memory descriptors were registered/published (init ordering bug), or descriptor registration failed silently.","commonSituations":"Race between scheduler issuing sends and the mori endpoint finishing descriptor exchange; failed startup registration swallowed by retries.","solutions":["Ensure mori endpoint init/registration completes before any send_kvcache call (check startup logs for descriptor publication)","Restart the prefill instance if registration was skipped","Verify allocator actually created KV pools (non-empty model load)"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"assert kv_mem_descs, \"mori descriptors not registered yet; defer send_kvcache\"","typeGuard":"def descriptors_ready(conn) -> bool: return len(conn.kv_mem_descs) > 0","tryCatchPattern":"catch RuntimeError 'KV memory descriptors are empty'; queue the send until descriptor registration completes rather than crashing","preventionTips":["Gate scheduler sends on a descriptor-registration readiness flag","Log registration completion prominently at startup"],"tags":["mori","kv-cache","descriptor","initialization-order"],"backgroundTag":"uninitialized-component","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}