{"record":{"id":"b19e7accdfbd55a9","repo":"sgl-project/sglang","slug":"failed-to-register-buffer-to-simm","errorCode":null,"errorMessage":"Failed to register buffer to SiMM","messagePattern":"Failed to register buffer to SiMM","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/mem_cache/storage/simm/hicache_simm.py","lineNumber":258,"sourceCode":"            logger.warning(f\"SiMM client warmup key {warmup_key} data wrong\")\n        logger.info(\n            f\"finish SiMM client warm up, cost {(time.perf_counter_ns() - start_time)/1000:.2f} us\"\n        )\n\n    def register_mem_pool_host(self, mem_pool_host: HostKVCache):\n        super().register_mem_pool_host(mem_pool_host)\n        assert self.mem_pool_host.layout in [\n            \"page_first\",\n            \"page_first_direct\",\n        ], \"simm storage backend only support page first or page first direct layout\"\n        buffer = self.mem_pool_host.kv_buffer\n        try:\n            self.mr_ext = register_mr(buffer)\n            if self.mr_ext is None:\n                logger.error(\n                    f\"Failed to register buffer, {buffer=}, please check buffer and RDMA network\"\n                )\n                raise RuntimeError(f\"Failed to register buffer to SiMM\")\n        except TypeError as err:\n            logger.error(\"Failed to register buffer to SiMM: %s\", err)\n            raise TypeError(\"SiMM Register Buffer Error.\") from err\n\n    def _get_mha_buffer_meta(self, keys, indices):\n        ptr_list, element_size_list = self.mem_pool_host.get_page_buffer_meta(indices)\n        key_list = []\n        for key_ in keys:\n            key_list.append(f\"{key_}_{self.mha_suffix}_k\")\n            key_list.append(f\"{key_}_{self.mha_suffix}_v\")\n        if len(key_list) != len(ptr_list):\n            logger.error(\n                f\"key size {len(key_list)} not equal with incides ptr size {len(ptr_list)}\"\n            )\n        assert len(key_list) == len(ptr_list)\n        return key_list, ptr_list, element_size_list\n\n    def _get_mla_buffer_meta(self, keys, indices):","sourceCodeStart":240,"sourceCodeEnd":276,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/storage/simm/hicache_simm.py#L240-L276","documentation":"Raised when registering a host memory buffer with SiMM RDMA fails: the underlying register_mr call returns None, meaning the memory region could not be pinned/registered with the RDMA subsystem. The code explicitly suggests checking the buffer and RDMA network in its log message.","triggerScenarios":"Calling SiMM.register_mem_pool_host(buffer) where register_mr(buffer) returns None — e.g. a non-contiguous/CPU tensor that can't be registered, missing RDMA devices, or a misconfigured/unstarted SiMM daemon.","commonSituations":"Host memory offload (HiCache) with --enable-hierarchical-cache plus RDMA transfer on a node without proper RDMA NICs, wrong IB devices, or passing a CUDA tensor instead of pinned host memory.","solutions":["Verify RDMA hardware/link: check ibv_devices, ensure the NIC and the SiMM/mori service are up","Ensure the buffer passed in is a valid pinned CPU tensor (not None, not a CUDA tensor)","Check register_mr logs for the underlying reason; align allocator/pinning settings with SiMM requirements","If RDMA is not needed, disable the SiMM/RDMA host-transfer backend and fall back to TCP/Posix"],"exampleFix":"// before\npool.register_mem_pool_host(cpu_tensor)\n// after\nimport torch\ncpu_tensor = cpu_tensor.pin_memory() if not cpu_tensor.is_pinned() else cpu_tensor\nassert cpu_tensor.device.type == 'cpu'\npool.register_mem_pool_host(cpu_tensor)","handlingStrategy":"validation","validationCode":"assert buffer is not None and getattr(buffer, 'device', None) is not None and buffer.device.type == 'cpu'\ntry:\n    buffer = buffer.pin_memory()\nexcept RuntimeError:\n    pass  # already pinned or not supported\n# check RDMA availability\nimport subprocess\nassert subprocess.run(['ibv_devices']).returncode == 0","typeGuard":"def is_registrable_host_buffer(b) -> bool:\n    import torch\n    return isinstance(b, torch.Tensor) and b.device.type == 'cpu' and not b.is_sparse","tryCatchPattern":"try:\n    store.register_mem_pool_host(buf)\nexcept RuntimeError as e:\n    if 'Failed to register buffer to SiMM' in str(e):\n        logger.fatal('RDMA registration failed; check NIC/buffer')\n        raise","preventionTips":["Pre-pin host pool tensors before registration","Run ibv_devices / verify SiMM daemon before enabling RDMA host transfer","Log buffer metadata (shape/dtype/device) at registration time"],"tags":["rdma","memory-registration","hierarchical-cache","sim"],"backgroundTag":"rdma-memory-registration-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}