{"record":{"id":"ae957cc82fc132ec","repo":"sgl-project/sglang","slug":"memory-size-memory-size-is-smaller-than-cuda-vmm","errorCode":null,"errorMessage":"memory_size={memory_size} is smaller than CUDA VMM granularity={granularity}","messagePattern":"memory_size=(.+?) is smaller than CUDA VMM granularity=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/cuda_vmm_transport_utils.py","lineNumber":291,"sourceCode":"            raise\n\n    @property\n    def fabric_handle(self) -> bytes | None:\n        return self.shareable_handle if self.use_fabric else None\n\n    def _allocate(self, memory_size: int) -> None:\n        drv = _get_cuda_driver()\n        prop = make_device_allocation_prop(\n            self.device_index,\n            handle_types=self.handle_type,\n            gpu_direct_rdma=self.use_fabric,\n        )\n\n        with torch.cuda.device(self.device_index):\n            granularity = get_allocation_granularity(prop)\n            allocation_size = memory_size // granularity * granularity\n            if allocation_size == 0:\n                raise ValueError(\n                    f\"memory_size={memory_size} is smaller than CUDA VMM \"\n                    f\"granularity={granularity}\"\n                )\n\n            allocation = VmmReservation(\n                allocation_size,\n                prop,\n                self.device_index,\n                alignment=granularity,\n            )\n            exported = None\n            try:\n                handle = allocation.map(\n                    0,\n                    allocation_size,\n                    retain_handle=True,\n                )\n                exported = check_drv(","sourceCodeStart":273,"sourceCodeEnd":309,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/cuda_vmm_transport_utils.py#L273-L309","documentation":"During _allocate, memory_size floors to 0 after rounding down to CUDA's allocation granularity (typically 2 MiB), meaning the requested size is smaller than one granularity unit and cannot be mapped.","triggerScenarios":"Passing a memory_size smaller than get_allocation_granularity() (e.g. a few KB/bytes) to the pool constructor.","commonSituations":"Small test pools, computed sizes based on tiny payloads, or unit conversions (MiB vs bytes) gone wrong.","solutions":["Size memory_size to at least the granularity (round up: align_up(size, granularity))","Check unit math — bytes vs MiB/GB","For tests, use a few MiB minimum"],"exampleFix":"# before\npool = CudaVmmTransportPool(memory_size=4096, ...)  # < 2 MiB granularity\n# after\npool = CudaVmmTransportPool(memory_size=2 * 1024 * 1024, ...)","handlingStrategy":"validation","validationCode":"gran = get_allocation_granularity(prop)\nif memory_size < gran:\n    memory_size = align_up(memory_size, gran)","typeGuard":null,"tryCatchPattern":"try:\n    pool = Pool(memory_size=size, ...)\nexcept ValueError as e:\n    if \"granularity\" in str(e):\n        size = align_up(size, 2 * 1024 * 1024); pool = Pool(memory_size=size, ...)","preventionTips":["Round pool sizes up to 2 MiB granularity","Double-check bytes vs MiB unit math"],"tags":["cuda","vmm","memory-allocation","granularity"],"backgroundTag":"memory-size-below-allocation-granularity","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}