sgl-project/sglang · critical · ImportError

cuda.bindings.driver is required for CUDA VMM operations

Error message

cuda.bindings.driver is required for CUDA VMM operations

What it means

The cuda_vmm modules lazily import cuda.bindings.driver; if the import failed at module load (package missing or incompatible version), _drv stays None and every VMM operation raises this ImportError. It indicates the environment lacks the required CUDA Python bindings, not a runtime CUDA failure.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_utils.py:57

    pynvml = None

_NVML_GPU_FABRIC_INFO_V3_TYPE = None
_NVML_GPU_FABRIC_INFO_V3_VERSION = None
if pynvml is not None:
    try:
        _NVML_GPU_FABRIC_INFO_V3_TYPE = pynvml.c_nvmlGpuFabricInfo_v3_t
        _NVML_GPU_FABRIC_INFO_V3_VERSION = pynvml.nvmlGpuFabricInfo_v3
    except AttributeError:
        pass

# NVML_GPU_FABRIC_STATE_COMPLETED: the GPU has joined its NVLink fabric clique.
_NVML_GPU_FABRIC_STATE_COMPLETED = 3


def _get_cuda_driver():
    """Return the imported CUDA driver bindings."""
    if _drv is None:
        raise ImportError("cuda.bindings.driver is required for CUDA VMM operations")
    return _drv


def check_drv(result_tuple, label):
    """Check a cuda.bindings driver call result and return the value."""
    if not isinstance(result_tuple, tuple):
        result_tuple = (result_tuple,)
    err = result_tuple[0]
    drv = _get_cuda_driver()
    if err != drv.CUresult.CUDA_SUCCESS:
        raise RuntimeError(f"{label}: {err}")
    return result_tuple[1] if len(result_tuple) > 1 else None


def tensor_from_pointer(
    pointer: int,
    nbytes: int,
    *,

View on GitHub (pinned to 0132848349)

Solutions

1. pip install cuda-python (or cuda-bindings) matching the environment's Python and CUDA version
2. Verify with: python -c "from cuda.bindings import driver" and fix any ImportError shown
3. If VMM features are optional, disable the code path (e.g. avoid cuda_vmm transport) until the package is installed

Example fix

# before: package missing
pip install sglang

# after
pip install cuda-python

try:
    from cuda.bindings import driver  # noqa
    ok = True
except ImportError:
    ok = False

def cuda_bindings_available() -> bool:
    try:
        from cuda.bindings import driver  # noqa
        return True
    except ImportError:
        return False

When it happens

Trigger: Any call to _allocate, is_vmm_pointer, compute_graph_capture_bases, make_rw_access_desc etc. without the cuda-python package installed; cuda-python installed for a different Python version so the import silently failed.

Common situations: Fresh environments or slim Docker images missing 'cuda-bindings'; pip resolving an old/broken nvidia-cuda-python-cu12; using a wheel of sglang that lists cuda-python as optional.

Related errors

• no supported CUDA VMM allocation handle type
• Can not import FA3 in sgl_kernel. Please check your installa
• GenerativeModel
• flash-attn is not installed. Please install it, e.g., `pip i
• MiniMax H3 AdaLN cache must be built on CUDA

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d26ae653135bda94. Report an issue: GitHub.