sgl-project/sglang · error · RuntimeError

CUDA VMM proxy has no shareable handle

Error message

CUDA VMM proxy has no shareable handle

What it means

Raised when building/importing a CUDA VMM pool on the consumer side and neither a fabric handle nor a POSIX socket path handle was provided. The proxy needs at least one OS-level shareable handle to import the exporter's virtual memory allocation; with none, import is impossible.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_transport_utils.py:667

            ]
        )


_imported_pool_cache: dict[tuple, _ImportedCudaVmmPool] = {}
_imported_pool_cache_lock = threading.Lock()


def _get_imported_pool(
    *,
    fabric_handle: bytes | None,
    posix_socket_path: str | None,
    allocation_size: int,
    device_index: int,
) -> _ImportedCudaVmmPool:
    use_fabric = fabric_handle is not None
    transport_handle = fabric_handle if use_fabric else posix_socket_path
    if transport_handle is None:
        raise RuntimeError("CUDA VMM proxy has no shareable handle")
    key = (device_index, allocation_size, transport_handle)
    pool = _imported_pool_cache.get(key)
    if pool is not None:
        return pool

    with _imported_pool_cache_lock:
        pool = _imported_pool_cache.get(key)
        if pool is not None:
            return pool

        fd = None
        try:
            if not use_fabric:
                fd = _receive_posix_fd(posix_socket_path)
            with torch.cuda.device(device_index):
                pointer = import_and_map_alloc(
                    fabric_handle,
                    fd,

View on GitHub (pinned to 0132848349)

Solutions

1. Verify the exporter side creates either a fabric or POSIX-socket handle for the allocation
2. Check the transport config/launch flags enable the handle mechanism appropriate for the fabric topology (POSIX for single host, fabric for multi-node)
3. Inspect the IPC message carrying the handles to confirm fields are populated before constructing the proxy

if fabric_handle is None and posix_socket_path is None:
    raise ValueError("cannot import VMM pool: no shareable handle provided")

def has_shareable_handle(proxy_msg) -> bool:
    return proxy_msg.fabric_handle is not None or proxy_msg.posix_socket_path is not None

When it happens

Trigger: Calling _get_imported_pool / _pool with fabric_handle=None and posix_socket_path=None; handle fields dropped during IPC serialization; running on a platform where fabric handle creation failed and the socket path was not set as fallback.

Common situations: Misconfigured multi-process setup where the exporter didn't create a shareable handle; gateway/NVSwitch-less machines where fabric handles are unavailable and the POSIX path fallback wasn't configured; serialization bug dropping handle attributes.

Related errors

• CUDA VMM POSIX FD broker failed
• CUDA VMM POSIX FD broker returned no file descriptor
• no supported CUDA VMM allocation handle type
• handle_types must be 'auto', an integer, or None
• invalid CUDA handle-type value: {handle_type_value}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bfdc302809cd7919. Report an issue: GitHub.