sgl-project/sglang · error · ValueError

invalid CUDA handle-type value: {handle_type_value}

Error message

invalid CUDA handle-type value: {handle_type_value}

What it means

After normalizing handle_types to an integer, the value is checked against the known set (NONE, POSIX, FABRIC, and whatever the installed bindings define). An integer outside this set — a stale bitmask, wrong combination, or typo'd constant — is rejected before building the CUmemAllocationProp.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_utils.py:297

    elif handle_types is None:
        handle_types = drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_NONE
    elif not isinstance(handle_types, int):
        raise ValueError("handle_types must be 'auto', an integer, or None")

    handle_type_value = int(handle_types)
    valid_handle_types = {
        int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_NONE): (
            drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_NONE
        ),
        int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR): (
            drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR
        ),
        int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_FABRIC): (
            drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_FABRIC
        ),
    }
    if handle_type_value not in valid_handle_types:
        raise ValueError(f"invalid CUDA handle-type value: {handle_type_value}")

    prop = drv.CUmemAllocationProp()
    prop.type = drv.CUmemAllocationType.CU_MEM_ALLOCATION_TYPE_PINNED
    prop.location.type = drv.CUmemLocationType.CU_MEM_LOCATION_TYPE_DEVICE
    prop.location.id = int(device_id)
    # cuda-bindings 13.0.x requires the generated enum here; newer releases
    # also accept a plain int, which previously hid this compatibility issue.
    prop.requestedHandleTypes = valid_handle_types[handle_type_value]
    prop.allocFlags.gpuDirectRDMACapable = int(gpu_direct_rdma)
    return prop


def get_allocation_granularity(prop, flag=_RECOMMENDED_GRANULARITY) -> int:
    """Return allocation granularity for a CUDA policy flag."""
    drv = _get_cuda_driver()
    return int(
        check_drv(
            drv.cuMemGetAllocationGranularity(prop, flag),

View on GitHub (pinned to 0132848349)

Solutions

1. Use int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_POSIX) (or the auto path) instead of magic numbers
2. Upgrade/downgrade cuda-python so its handle-type enum matches the constants you compute
3. Validate the bitmask against the installed bindings' enum values before calling

Example fix

# before
make_device_allocation_prop(0, handle_types=0x03)  # stale bitmask

# after
make_device_allocation_prop(0, handle_types='auto')

drv = _get_cuda_driver()
valid = {int(v) for k, v in vars(drv.CUmemAllocationHandleType).items() if k.startswith('CU_MEM_HANDLE_TYPE')}
assert int(handle_types) in valid

def is_known_handle_type_value(value: int) -> bool:
    drv = _get_cuda_driver()
    return value in {
        int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_NONE),
        int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_POSIX),
        int(drv.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_FABRIC),
    }

When it happens

Trigger: Passing a bitmask from a different CUDA version that includes bits the installed cuda-bindings doesn't define; computing handle types with bitwise-OR of deprecated flags; hardcoding a magic number.

Common situations: Code written against a newer/older CUDA header reused with different cuda-python version; OR-ing FABRIC and POSIX which may yield an unsupported combined value on some versions.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• handle_types must be 'auto', an integer, or None
• memory_size must be positive
• consumer_count must be positive
• recycle_interval must be positive
• CUDA VMM proxy has no shareable handle

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1528278d66e4138f. Report an issue: GitHub.