sgl-project/sglang · error · TypeError

SiMM Register Buffer Error.

Error message

SiMM Register Buffer Error.

What it means

A TypeError raised while registering a host buffer to SiMM — the register_mr call itself raised TypeError, typically because the buffer has an unexpected type/shape unsupported by the SiMM registration API. The original exception is chained via 'from err'.

Source

Thrown at python/sglang/srt/mem_cache/storage/simm/hicache_simm.py:261

        )

    def register_mem_pool_host(self, mem_pool_host: HostKVCache):
        super().register_mem_pool_host(mem_pool_host)
        assert self.mem_pool_host.layout in [
            "page_first",
            "page_first_direct",
        ], "simm storage backend only support page first or page first direct layout"
        buffer = self.mem_pool_host.kv_buffer
        try:
            self.mr_ext = register_mr(buffer)
            if self.mr_ext is None:
                logger.error(
                    f"Failed to register buffer, {buffer=}, please check buffer and RDMA network"
                )
                raise RuntimeError(f"Failed to register buffer to SiMM")
        except TypeError as err:
            logger.error("Failed to register buffer to SiMM: %s", err)
            raise TypeError("SiMM Register Buffer Error.") from err

    def _get_mha_buffer_meta(self, keys, indices):
        ptr_list, element_size_list = self.mem_pool_host.get_page_buffer_meta(indices)
        key_list = []
        for key_ in keys:
            key_list.append(f"{key_}_{self.mha_suffix}_k")
            key_list.append(f"{key_}_{self.mha_suffix}_v")
        if len(key_list) != len(ptr_list):
            logger.error(
                f"key size {len(key_list)} not equal with incides ptr size {len(ptr_list)}"
            )
        assert len(key_list) == len(ptr_list)
        return key_list, ptr_list, element_size_list

    def _get_mla_buffer_meta(self, keys, indices):
        ptr_list, element_size_list = self.mem_pool_host.get_page_buffer_meta(indices)
        key_list = []
        for key_ in keys:

View on GitHub (pinned to 0132848349)

Solutions

1. Check the logged chained error ('Failed to register buffer to SiMM: %s') for the exact argument mismatch
2. Match mori/SiMM version to the sglang release requirements and reinstall
3. Ensure the buffer is a torch CPU tensor allocated by the host mem pool allocator
4. If the type is genuinely unsupported upstream, report/fallback to the default host allocator

Example fix

// before
store.register_mem_pool_host(np_array)
// after
store.register_mem_pool_host(torch.from_numpy(np_array))  # torch CPU tensor expected

import torch
assert isinstance(buffer, torch.Tensor), f'expected torch.Tensor, got {type(buffer)}'

def is_supported_buffer_type(b) -> bool:
    import torch
    return isinstance(b, torch.Tensor)

try:
    store.register_mem_pool_host(buf)
except TypeError as e:
    if 'SiMM Register Buffer' in str(e):
        buf = torch.as_tensor(buf)
        store.register_mem_pool_host(buf)  # retry with converted type

When it happens

Trigger: Calling register_mem_pool_host with a buffer whose type register_mr does not accept (e.g. a numpy array, list, or tensor with unsupported dtype/layout), producing a TypeError inside register_mr.

Common situations: Version mismatch between the installed mori/SiMM wheel and sglang's expected buffer API; passing a pool buffer allocated with a non-standard allocator.

Understand the failure class

Background: Invalid argument type errors: "must be of type string", "expected X, got Y", and ERR_INVALID_ARG_TYPE explained — this error's family across 15 libraries.

Related errors

• Failed to register buffer to SiMM
• NIXL memory registration failed for {mem_kind} kv tensors
• Failed to register buffer to Mooncake Store, error code: {re
• spt must be a bool when provided
• Unknown type: {type(other)}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a1eecdf1a089a1d2. Report an issue: GitHub.