{"record":{"id":"b709552dd46bb19d","repo":"sgl-project/sglang","slug":"no-supported-cuda-vmm-allocation-handle-type","errorCode":null,"errorMessage":"no supported CUDA VMM allocation handle type","messagePattern":"no supported CUDA VMM allocation handle type","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/utils/cuda_vmm_utils.py","lineNumber":266,"sourceCode":"                drv.cuMemRelease(probe_handle),\n                f\"cuMemRelease({name} probe)\",\n            )\n        except RuntimeError as error:\n            last_error = error\n            logger.warning(\n                \"CUDA VMM %s backing unavailable on device %d; trying fallback: %s\",\n                name,\n                device_id,\n                error,\n            )\n            continue\n        logger.info(\n            \"CUDA VMM selected %s backing for device %d\",\n            name,\n            device_id,\n        )\n        return handle_type\n    raise RuntimeError(\"no supported CUDA VMM allocation handle type\") from last_error\n\n\ndef make_device_allocation_prop(\n    device_id: int,\n    *,\n    handle_types: int | str | None = \"auto\",\n    gpu_direct_rdma: bool = False,\n):\n    \"\"\"Build a device allocation prop with automatic or explicit exportability.\"\"\"\n    drv = _get_cuda_driver()\n    if handle_types == \"auto\":\n        handle_types = get_device_allocation_handle_type(device_id)\n    elif handle_types is None:\n        handle_types = drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_NONE\n    elif not isinstance(handle_types, int):\n        raise ValueError(\"handle_types must be 'auto', an integer, or None\")\n\n    handle_type_value = int(handle_types)","sourceCodeStart":248,"sourceCodeEnd":284,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/cuda_vmm_utils.py#L248-L284","documentation":"get_device_allocation_handle_type probes candidate CUmemAllocationHandleType values for a device and all failed, so no supported way to share VMM allocations (POSIX fd, fabric, etc.) exists. The last probe error is chained. This is an environment/capability failure: the device or driver cannot produce shareable memory handles.","triggerScenarios":"Running VMM transport on a device/driver without required handle support (e.g. no fabric on non-NVLink systems and POSIX handle creation failing); old driver lacking cuMemExportToShareableHandle; restrictive container blocking fd creation.","commonSituations":"Older datacenter or consumer GPUs with outdated drivers; docker containers with restricted /dev shm or fd limits; MIG instances where some handle types are unsupported.","solutions":["Update the NVIDIA driver to a version supporting VMM shareable handles (R515+)","Check container runtime settings: sufficient fd limit and /dev/shm access for POSIX handle types","Fall back to non-VMM feature transport if the hardware/driver can't support shared handles"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"try:\n    handle_type = get_device_allocation_handle_type(device_id)\nexcept RuntimeError:\n    handle_type = None  # no sharing possible on this device","typeGuard":"def vmm_handle_supported(device_id: int) -> bool:\n    try:\n        get_device_allocation_handle_type(device_id)\n        return True\n    except RuntimeError:\n        return False","tryCatchPattern":null,"preventionTips":["Probe handle support at startup and fall back to default transport","Keep NVIDIA drivers current (R515+) in VMM deployments"],"tags":["cuda","vmm","handle","driver-capability","environment"],"backgroundTag":"unsupported-driver-capability","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}