sgl-project/sglang · critical · RuntimeError

no supported CUDA VMM allocation handle type

Error message

no supported CUDA VMM allocation handle type

What it means

get_device_allocation_handle_type probes candidate CUmemAllocationHandleType values for a device and all failed, so no supported way to share VMM allocations (POSIX fd, fabric, etc.) exists. The last probe error is chained. This is an environment/capability failure: the device or driver cannot produce shareable memory handles.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_utils.py:266

                drv.cuMemRelease(probe_handle),
                f"cuMemRelease({name} probe)",
            )
        except RuntimeError as error:
            last_error = error
            logger.warning(
                "CUDA VMM %s backing unavailable on device %d; trying fallback: %s",
                name,
                device_id,
                error,
            )
            continue
        logger.info(
            "CUDA VMM selected %s backing for device %d",
            name,
            device_id,
        )
        return handle_type
    raise RuntimeError("no supported CUDA VMM allocation handle type") from last_error


def make_device_allocation_prop(
    device_id: int,
    *,
    handle_types: int | str | None = "auto",
    gpu_direct_rdma: bool = False,
):
    """Build a device allocation prop with automatic or explicit exportability."""
    drv = _get_cuda_driver()
    if handle_types == "auto":
        handle_types = get_device_allocation_handle_type(device_id)
    elif handle_types is None:
        handle_types = drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_NONE
    elif not isinstance(handle_types, int):
        raise ValueError("handle_types must be 'auto', an integer, or None")

    handle_type_value = int(handle_types)

View on GitHub (pinned to 0132848349)

Solutions

1. Update the NVIDIA driver to a version supporting VMM shareable handles (R515+)
2. Check container runtime settings: sufficient fd limit and /dev/shm access for POSIX handle types
3. Fall back to non-VMM feature transport if the hardware/driver can't support shared handles

try:
    handle_type = get_device_allocation_handle_type(device_id)
except RuntimeError:
    handle_type = None  # no sharing possible on this device

def vmm_handle_supported(device_id: int) -> bool:
    try:
        get_device_allocation_handle_type(device_id)
        return True
    except RuntimeError:
        return False

When it happens

Trigger: Running VMM transport on a device/driver without required handle support (e.g. no fabric on non-NVLink systems and POSIX handle creation failing); old driver lacking cuMemExportToShareableHandle; restrictive container blocking fd creation.

Common situations: Older datacenter or consumer GPUs with outdated drivers; docker containers with restricted /dev shm or fd limits; MIG instances where some handle types are unsupported.

Related errors

• CUDA VMM proxy has no shareable handle
• cuda.bindings.driver is required for CUDA VMM operations
• handle_types must be 'auto', an integer, or None
• invalid CUDA handle-type value: {handle_type_value}
• MiniMax H3 AdaLN cache must be built on CUDA

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b709552dd46bb19d. Report an issue: GitHub.