sgl-project/sglang · error · ValueError

handle_types must be 'auto', an integer, or None

Error message

handle_types must be 'auto', an integer, or None

What it means

make_device_allocation_prop accepts handle_types as the string 'auto' (auto-detect), None (no sharing), or a raw integer handle-type bitmask. Any other type — a list, a string other than 'auto', an enum object on some versions — raises this ValueError before touching the driver.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_utils.py:282

        )
        return handle_type
    raise RuntimeError("no supported CUDA VMM allocation handle type") from last_error


def make_device_allocation_prop(
    device_id: int,
    *,
    handle_types: int | str | None = "auto",
    gpu_direct_rdma: bool = False,
):
    """Build a device allocation prop with automatic or explicit exportability."""
    drv = _get_cuda_driver()
    if handle_types == "auto":
        handle_types = get_device_allocation_handle_type(device_id)
    elif handle_types is None:
        handle_types = drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_NONE
    elif not isinstance(handle_types, int):
        raise ValueError("handle_types must be 'auto', an integer, or None")

    handle_type_value = int(handle_types)
    valid_handle_types = {
        int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_NONE): (
            drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_NONE
        ),
        int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR): (
            drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR
        ),
        int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_FABRIC): (
            drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_FABRIC
        ),
    }
    if handle_type_value not in valid_handle_types:
        raise ValueError(f"invalid CUDA handle-type value: {handle_type_value}")

    prop = drv.CUmemAllocationProp()
    prop.type = drv.CUmemAllocationType.CU_MEM_ALLOCATION_TYPE_PINNED

View on GitHub (pinned to 0132848349)

Solutions

1. Pass int(handle_types) — convert enums with int() before the call
2. Use 'auto' to let the library detect the best handle type, or None for no sharing
3. Map config strings to int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_POSIX) etc. at your config layer

Example fix

# before
prop = make_device_allocation_prop(0, handle_types="posix")

# after
from cuda.bindings import driver as drv
prop = make_device_allocation_prop(
    0, handle_types=int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_POSIX)
)

assert handle_types == "auto" or handle_types is None or isinstance(handle_types, int)

def valid_handle_types(h) -> bool:
    return h == "auto" or h is None or isinstance(h, int) and not isinstance(h, bool)

When it happens

Trigger: Passing handle_types=['posix'] or 'fabric' (string names) instead of the integer bitmask; passing a CUmemAllocationHandleType enum instance where the installed cuda-bindings requires int; passing a float.

Common situations: User config exposing handle type names that aren't converted to int bitmasks; version upgrades of cuda-bindings changing enum/int acceptance.

Understand the failure class

Background: Invalid argument type errors: "must be of type string", "expected X, got Y", and ERR_INVALID_ARG_TYPE explained — this error's family across 15 libraries.

Related errors

• CUDA VMM feature transport requires each feature field to co
• invalid CUDA handle-type value: {handle_type_value}
• memory_size must be positive
• consumer_count must be positive
• recycle_interval must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/82c0a5ce2776b072. Report an issue: GitHub.