sgl-project/sglang · error · ValueError

memory_size={memory_size} is smaller than CUDA VMM granulari

Error message

memory_size={memory_size} is smaller than CUDA VMM granularity={granularity}

What it means

During _allocate, memory_size floors to 0 after rounding down to CUDA's allocation granularity (typically 2 MiB), meaning the requested size is smaller than one granularity unit and cannot be mapped.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_transport_utils.py:291

            raise

    @property
    def fabric_handle(self) -> bytes | None:
        return self.shareable_handle if self.use_fabric else None

    def _allocate(self, memory_size: int) -> None:
        drv = _get_cuda_driver()
        prop = make_device_allocation_prop(
            self.device_index,
            handle_types=self.handle_type,
            gpu_direct_rdma=self.use_fabric,
        )

        with torch.cuda.device(self.device_index):
            granularity = get_allocation_granularity(prop)
            allocation_size = memory_size // granularity * granularity
            if allocation_size == 0:
                raise ValueError(
                    f"memory_size={memory_size} is smaller than CUDA VMM "
                    f"granularity={granularity}"
                )

            allocation = VmmReservation(
                allocation_size,
                prop,
                self.device_index,
                alignment=granularity,
            )
            exported = None
            try:
                handle = allocation.map(
                    0,
                    allocation_size,
                    retain_handle=True,
                )
                exported = check_drv(

View on GitHub (pinned to 0132848349)

Solutions

1. Size memory_size to at least the granularity (round up: align_up(size, granularity))
2. Check unit math — bytes vs MiB/GB
3. For tests, use a few MiB minimum

Example fix

# before
pool = CudaVmmTransportPool(memory_size=4096, ...)  # < 2 MiB granularity
# after
pool = CudaVmmTransportPool(memory_size=2 * 1024 * 1024, ...)

gran = get_allocation_granularity(prop)
if memory_size < gran:
    memory_size = align_up(memory_size, gran)

try:
    pool = Pool(memory_size=size, ...)
except ValueError as e:
    if "granularity" in str(e):
        size = align_up(size, 2 * 1024 * 1024); pool = Pool(memory_size=size, ...)

When it happens

Trigger: Passing a memory_size smaller than get_allocation_granularity() (e.g. a few KB/bytes) to the pool constructor.

Common situations: Small test pools, computed sizes based on tiny payloads, or unit conversions (MiB vs bytes) gone wrong.

Related errors

• CUDA error: {err}
• CUDA VMM POSIX FD broker failed
• CUDA VMM POSIX FD broker returned no file descriptor
• memory_size must be positive
• consumer_count must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ae957cc82fc132ec. Report an issue: GitHub.