{"record":{"id":"76fe67d3846aeb96","repo":"sgl-project/sglang","slug":"umbphosttensorallocator-only-supports-cpu-host-mem","errorCode":null,"errorMessage":"UMBPHostTensorAllocator only supports CPU host memory, got device={}","messagePattern":"UMBPHostTensorAllocator only supports CPU host memory, got device=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/storage/umbp/umbp_host_allocator.py","lineNumber":54,"sourceCode":"                \"or fall back to the default torch host allocator.\"\n            ) from exc\n\n        self._mod = umbp_mod\n        self._allocator = umbp_mod.UMBPHostMemAllocator()\n\n        self._use_hugepage = _bool_env(\"SGLANG_HICACHE_HOST_HUGEPAGE\", True)\n        self._hugepage_size = _int_env(\n            \"SGLANG_HICACHE_HOST_HUGEPAGE_SIZE\", 2 * 1024 * 1024\n        )\n        self._numa_node = _int_env(\"SGLANG_HICACHE_HOST_NUMA_NODE\", -1)\n        self._prefault = _bool_env(\"SGLANG_HICACHE_HOST_PREFAULT\", True)\n        self._handles: Dict[int, Any] = {}\n\n    def allocate(\n        self, dims: tuple, dtype: torch.dtype, device: str = \"cpu\"\n    ) -> torch.Tensor:\n        if device != \"cpu\":\n            raise ValueError(\n                \"UMBPHostTensorAllocator only supports CPU host memory, \"\n                f\"got device={device}\"\n            )\n\n        self.dims = dims\n        self.dtype = dtype\n\n        element_size = torch.empty((), dtype=dtype).element_size()\n        nbytes = math.prod(int(dim) for dim in dims) * element_size\n\n        requested_backing = (\n            self._mod.UMBPHostBufferBacking.AnonymousHugetlb\n            if self._use_hugepage\n            else self._mod.UMBPHostBufferBacking.Anonymous\n        )\n\n        handle = self._allocator.alloc(\n            nbytes,","sourceCodeStart":36,"sourceCodeEnd":72,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/storage/umbp/umbp_host_allocator.py#L36-L72","documentation":"UMBPHostTensorAllocator.allocate only manages CPU host memory; passing any other device string is rejected with this ValueError listing the offending device.","triggerScenarios":"Calling allocate(dims, dtype, device='cuda') or any device != 'cpu'.","commonSituations":"Generic tensor-allocator code reused across GPU and host pools forwards a 'cuda' device into the UMBP host allocator.","solutions":["Call allocate with device='cpu' (or omit it) for host memory","Allocate GPU tensors with the GPU allocator, not this class"],"exampleFix":"# before\nt = host_allocator.allocate((n, d), dtype, device='cuda')\n# after\nt = host_allocator.allocate((n, d), dtype, device='cpu')","handlingStrategy":"type-guard","validationCode":"device = device or 'cpu'\nassert device == 'cpu', 'host allocator is CPU-only; use the GPU allocator for device tensors'","typeGuard":"def is_cpu_device(device) -> bool:\n    return device in ('cpu', None) or str(device).startswith('cpu')","tryCatchPattern":null,"preventionTips":["Route allocations by device at the call site","Name allocator variables clearly (host_alloc vs gpu_alloc)"],"tags":["umbp","device-mismatch","validation"],"backgroundTag":"invalid-argument-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}