{"record":{"id":"cf9e440f414671d1","repo":"sgl-project/sglang","slug":"umbphostmemallocator-alloc-bytes-failed-reque","errorCode":null,"errorMessage":"UMBPHostMemAllocator.alloc({} bytes) failed (requested_backing={}, numa_node={}).","messagePattern":"UMBPHostMemAllocator\\.alloc\\((.+?) bytes\\) failed \\(requested_backing=(.+?), numa_node=(.+?)\\)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/mem_cache/storage/umbp/umbp_host_allocator.py","lineNumber":79,"sourceCode":"\n        element_size = torch.empty((), dtype=dtype).element_size()\n        nbytes = math.prod(int(dim) for dim in dims) * element_size\n\n        requested_backing = (\n            self._mod.UMBPHostBufferBacking.AnonymousHugetlb\n            if self._use_hugepage\n            else self._mod.UMBPHostBufferBacking.Anonymous\n        )\n\n        handle = self._allocator.alloc(\n            nbytes,\n            requested_backing,\n            self._hugepage_size,\n            self._numa_node,\n            self._prefault,\n        )\n        if not handle:\n            raise RuntimeError(\n                f\"UMBPHostMemAllocator.alloc({nbytes} bytes) failed \"\n                f\"(requested_backing={requested_backing}, \"\n                f\"numa_node={self._numa_node}).\"\n            )\n        self._handles[int(handle.ptr)] = handle\n\n        c_array = (ctypes.c_byte * nbytes).from_address(handle.ptr)\n        tensor = torch.frombuffer(c_array, dtype=torch.uint8, count=nbytes)\n\n        if dtype != torch.uint8:\n            tensor = tensor.view(dtype)\n\n        logger.info(\n            \"UMBPHostTensorAllocator: allocated %.2f GB at 0x%x \"\n            \"requested_backing=%s actual_backing=%s actual_alignment=%d \"\n            \"mapped_size=%d numa_node=%d\",\n            nbytes / 1e9,\n            handle.ptr,","sourceCodeStart":61,"sourceCodeEnd":97,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/storage/umbp/umbp_host_allocator.py#L61-L97","documentation":"The native UMBPHostMemAllocator.alloc returned a null/empty handle for the requested byte count. The message includes nbytes, requested backing size (hugepage-rounded) and NUMA node, which are the usual culprits.","triggerScenarios":"Requesting an allocation whose hugepage-rounded backing exceeds available hugepages/NUMA capacity, or NUMA node id out of range, so alloc returns an falsy handle.","commonSituations":"Oversized hierarchical-cache host pool with insufficient hugepages configured (nr_hugepages), wrong NUMA node on a single-node machine, or prefault exhausting memory.","solutions":["Check hugepage availability: cat /proc/meminfo | grep -i huge; increase nr_hugepages via sysctl","Reduce the host KV cache size so the rounded backing fits","Verify the numa_node config matches the machine topology (or use -1/auto)","Check dmesg/OOM logs for hugepage allocation failures and free memory"],"exampleFix":"# before\ncfg.numa_node = 3   # machine only has 2 nodes\n# after\ncfg.numa_node = 0   # valid node with free hugepages\nsysctl -w vm.nr_hugepages=4096","handlingStrategy":"validation","validationCode":"import os\ndef hugepages_free():\n    for line in open('/proc/meminfo'):\n        if line.startswith('HugePages_Free'):\n            return int(line.split()[1])\npage = 2048  # kB, verify\nneeded = ((nbytes + page*1024 - 1)//(page*1024))\nassert hugepages_free() >= needed, 'insufficient hugepages for UMBP allocation'","typeGuard":null,"tryCatchPattern":"try:\n    t = alloc.allocate(dims, dtype)\nexcept RuntimeError as e:\n    if 'UMBPHostMemAllocator.alloc' in str(e):\n        reduce_pool_size(); restart_or_retry()","preventionTips":["Pre-provision hugepages at boot (vm.nr_hugepages) with headroom over the configured host cache size","Validate numa_node against /sys/devices/system/node/ before starting","Size host cache to hugepage-rounded backing, not raw nbytes"],"tags":["umbp","hugepages","numa","out-of-memory"],"backgroundTag":"memory-allocation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}