sgl-project/sglang · error · Exception

Invalid device_uuid=

Error message

Invalid device_uuid=

What it means

Raised by the torch patching layer when a string passed as a device identifier does not match any visible CUDA device's UUID. The patch iterates torch.cuda.device_count() devices comparing str(props.uuid) to the input; no match means the UUID is malformed or the GPU is not visible to this process.

Source

Thrown at python/sglang/srt/utils/patch_torch.py:97

def _rebuild_cuda_tensor_modified(*args):
    args = _modify_tuple(args, _REDUCE_TENSOR_ARG_DEVICE_INDEX, _device_from_maybe_uuid)
    return reductions._rebuild_cuda_tensor_original(*args)


def _device_to_uuid(device: int) -> str:
    return str(torch.cuda.get_device_properties(device).uuid)


def _device_from_maybe_uuid(device_maybe_uuid: Union[int, str]) -> int:
    if isinstance(device_maybe_uuid, int):
        return device_maybe_uuid

    if isinstance(device_maybe_uuid, str):
        for device in range(torch.cuda.device_count()):
            if str(torch.cuda.get_device_properties(device).uuid) == device_maybe_uuid:
                return device
        raise Exception("Invalid device_uuid=" + device_maybe_uuid)

    raise Exception(f"Unknown type: {device_maybe_uuid=}")


def _modify_tuple(t, index: int, modifier: Callable):
    return *t[:index], modifier(t[index]), *t[index + 1 :]


def monkey_patch_torch_compile():
    if torch_release < (2, 8):
        # These things are cacheable by torch.compile. torch.compile just doesn't know it.
        # This was fixed in PyTorch 2.8, but until then, we monkey patch.
        import torch._higher_order_ops.auto_functionalize as af

        af.auto_functionalized_v2._cacheable = True
        af.auto_functionalized._cacheable = True

View on GitHub (pinned to 0132848349)

Solutions

1. Verify the UUID with nvidia-smi -L and paste the full 'GPU-xxxxxxxx-...' string
2. Ensure the target GPU is visible: unset/fix CUDA_VISIBLE_DEVICES for the process
3. Use a plain integer device index instead of a UUID

Example fix

# before
device_uuid='GPU-abc123'  # typo/truncated
# after
device_uuid='GPU-3d2f7a1b-09c4-11ee-be56-0242ac120002'  # full UUID from nvidia-smi -L

import subprocess
def uuid_exists(uuid: str) -> bool:
    out = subprocess.run(['nvidia-smi','-L'], capture_output=True, text=True).stdout
    return uuid in out

def is_valid_device_spec(v) -> bool:
    return isinstance(v, (int, str)) or v is None

try:
    idx = _device_from_maybe_uuid(uuid)
except Exception as e:
    if 'Invalid device_uuid' in str(e):
        idx = 0  # fallback device

When it happens

Trigger: Passing device_uuid='GPU-<wrong-or-clipped-uuid>' via config (e.g. CUDA_VISIBLE_DEVICES UUID form or a server arg) while the referenced GPU is masked by CUDA_VISIBLE_DEVICES by index, or the UUID string is truncated/malformed.

Common situations: Multi-node clusters addressing GPUs by UUID; NVIDIA MIG instances; typo'd or copy-truncated UUIDs; GPUs hidden by cgroup/device visibility.

Related errors

• Unknown type: {device_maybe_uuid=}
• indices must be on q's device {device}, got {indices.device}
• CUDART error: {error_str}
• NCCL only supports CUDA, ROCm and MUSA backends.
• Decode context parallel (decode_context_parallel_size > 1) i

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b14501453a6296cb. Report an issue: GitHub.