sgl-project/sglang · critical · RuntimeError

UMBPHostMemAllocator.alloc({} bytes) failed (requested_backi

Error message

UMBPHostMemAllocator.alloc({} bytes) failed (requested_backing={}, numa_node={}).

What it means

The native UMBPHostMemAllocator.alloc returned a null/empty handle for the requested byte count. The message includes nbytes, requested backing size (hugepage-rounded) and NUMA node, which are the usual culprits.

Source

Thrown at python/sglang/srt/mem_cache/storage/umbp/umbp_host_allocator.py:79

        element_size = torch.empty((), dtype=dtype).element_size()
        nbytes = math.prod(int(dim) for dim in dims) * element_size

        requested_backing = (
            self._mod.UMBPHostBufferBacking.AnonymousHugetlb
            if self._use_hugepage
            else self._mod.UMBPHostBufferBacking.Anonymous
        )

        handle = self._allocator.alloc(
            nbytes,
            requested_backing,
            self._hugepage_size,
            self._numa_node,
            self._prefault,
        )
        if not handle:
            raise RuntimeError(
                f"UMBPHostMemAllocator.alloc({nbytes} bytes) failed "
                f"(requested_backing={requested_backing}, "
                f"numa_node={self._numa_node})."
            )
        self._handles[int(handle.ptr)] = handle

        c_array = (ctypes.c_byte * nbytes).from_address(handle.ptr)
        tensor = torch.frombuffer(c_array, dtype=torch.uint8, count=nbytes)

        if dtype != torch.uint8:
            tensor = tensor.view(dtype)

        logger.info(
            "UMBPHostTensorAllocator: allocated %.2f GB at 0x%x "
            "requested_backing=%s actual_backing=%s actual_alignment=%d "
            "mapped_size=%d numa_node=%d",
            nbytes / 1e9,
            handle.ptr,

View on GitHub (pinned to 0132848349)

Solutions

Check hugepage availability: cat /proc/meminfo | grep -i huge; increase nr_hugepages via sysctl
Reduce the host KV cache size so the rounded backing fits
Verify the numa_node config matches the machine topology (or use -1/auto)
Check dmesg/OOM logs for hugepage allocation failures and free memory

Example fix

# before
cfg.numa_node = 3   # machine only has 2 nodes
# after
cfg.numa_node = 0   # valid node with free hugepages
sysctl -w vm.nr_hugepages=4096

Defensive patterns

Strategy: validation

Validate before calling

import os
def hugepages_free():
    for line in open('/proc/meminfo'):
        if line.startswith('HugePages_Free'):
            return int(line.split()[1])
page = 2048  # kB, verify
needed = ((nbytes + page*1024 - 1)//(page*1024))
assert hugepages_free() >= needed, 'insufficient hugepages for UMBP allocation'

Try / catch

try:
    t = alloc.allocate(dims, dtype)
except RuntimeError as e:
    if 'UMBPHostMemAllocator.alloc' in str(e):
        reduce_pool_size(); restart_or_retry()

Prevention

Pre-provision hugepages at boot (vm.nr_hugepages) with headroom over the configured host cache size
Validate numa_node against /sys/devices/system/node/ before starting
Size host cache to hugepage-rounded backing, not raw nbytes

When it happens

Trigger: Requesting an allocation whose hugepage-rounded backing exceeds available hugepages/NUMA capacity, or NUMA node id out of range, so alloc returns an falsy handle.

Common situations: Oversized hierarchical-cache host pool with insufficient hugepages configured (nr_hugepages), wrong NUMA node on a single-node machine, or prefault exhausting memory.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cf9e440f414671d1. Report an issue: GitHub.