sgl-project/sglang · critical · RuntimeError

Failed to register buffer to SiMM

Error message

Failed to register buffer to SiMM

What it means

Raised when registering a host memory buffer with SiMM RDMA fails: the underlying register_mr call returns None, meaning the memory region could not be pinned/registered with the RDMA subsystem. The code explicitly suggests checking the buffer and RDMA network in its log message.

Source

Thrown at python/sglang/srt/mem_cache/storage/simm/hicache_simm.py:258

            logger.warning(f"SiMM client warmup key {warmup_key} data wrong")
        logger.info(
            f"finish SiMM client warm up, cost {(time.perf_counter_ns() - start_time)/1000:.2f} us"
        )

    def register_mem_pool_host(self, mem_pool_host: HostKVCache):
        super().register_mem_pool_host(mem_pool_host)
        assert self.mem_pool_host.layout in [
            "page_first",
            "page_first_direct",
        ], "simm storage backend only support page first or page first direct layout"
        buffer = self.mem_pool_host.kv_buffer
        try:
            self.mr_ext = register_mr(buffer)
            if self.mr_ext is None:
                logger.error(
                    f"Failed to register buffer, {buffer=}, please check buffer and RDMA network"
                )
                raise RuntimeError(f"Failed to register buffer to SiMM")
        except TypeError as err:
            logger.error("Failed to register buffer to SiMM: %s", err)
            raise TypeError("SiMM Register Buffer Error.") from err

    def _get_mha_buffer_meta(self, keys, indices):
        ptr_list, element_size_list = self.mem_pool_host.get_page_buffer_meta(indices)
        key_list = []
        for key_ in keys:
            key_list.append(f"{key_}_{self.mha_suffix}_k")
            key_list.append(f"{key_}_{self.mha_suffix}_v")
        if len(key_list) != len(ptr_list):
            logger.error(
                f"key size {len(key_list)} not equal with incides ptr size {len(ptr_list)}"
            )
        assert len(key_list) == len(ptr_list)
        return key_list, ptr_list, element_size_list

    def _get_mla_buffer_meta(self, keys, indices):

View on GitHub (pinned to 0132848349)

Solutions

1. Verify RDMA hardware/link: check ibv_devices, ensure the NIC and the SiMM/mori service are up
2. Ensure the buffer passed in is a valid pinned CPU tensor (not None, not a CUDA tensor)
3. Check register_mr logs for the underlying reason; align allocator/pinning settings with SiMM requirements
4. If RDMA is not needed, disable the SiMM/RDMA host-transfer backend and fall back to TCP/Posix

Example fix

// before
pool.register_mem_pool_host(cpu_tensor)
// after
import torch
cpu_tensor = cpu_tensor.pin_memory() if not cpu_tensor.is_pinned() else cpu_tensor
assert cpu_tensor.device.type == 'cpu'
pool.register_mem_pool_host(cpu_tensor)

assert buffer is not None and getattr(buffer, 'device', None) is not None and buffer.device.type == 'cpu'
try:
    buffer = buffer.pin_memory()
except RuntimeError:
    pass  # already pinned or not supported
# check RDMA availability
import subprocess
assert subprocess.run(['ibv_devices']).returncode == 0

def is_registrable_host_buffer(b) -> bool:
    import torch
    return isinstance(b, torch.Tensor) and b.device.type == 'cpu' and not b.is_sparse

try:
    store.register_mem_pool_host(buf)
except RuntimeError as e:
    if 'Failed to register buffer to SiMM' in str(e):
        logger.fatal('RDMA registration failed; check NIC/buffer')
        raise

When it happens

Trigger: Calling SiMM.register_mem_pool_host(buffer) where register_mr(buffer) returns None — e.g. a non-contiguous/CPU tensor that can't be registered, missing RDMA devices, or a misconfigured/unstarted SiMM daemon.

Common situations: Host memory offload (HiCache) with --enable-hierarchical-cache plus RDMA transfer on a node without proper RDMA NICs, wrong IB devices, or passing a CUDA tensor instead of pinned host memory.

Related errors

• SiMM Register Buffer Error.
• NIXL memory registration failed for {mem_kind} kv tensors
• Failed to register buffer to Mooncake Store, error code: {re
• MiniCPM SALA does not support hierarchical cache
• PD decode DCP currently requires chunk cache; --enable-hiera

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b19e7accdfbd55a9. Report an issue: GitHub.