sgl-project/sglang · error · Exception

Unknown type: {device_maybe_uuid=}

Error message

Unknown type: {device_maybe_uuid=}

What it means

The device-patching helper received a value that is neither a torch.device/int-like device nor a str, so it cannot resolve it to a CUDA device index and raises a generic Exception.

Source

Thrown at python/sglang/srt/utils/patch_torch.py:99

    args = _modify_tuple(args, _REDUCE_TENSOR_ARG_DEVICE_INDEX, _device_from_maybe_uuid)
    return reductions._rebuild_cuda_tensor_original(*args)


def _device_to_uuid(device: int) -> str:
    return str(torch.cuda.get_device_properties(device).uuid)


def _device_from_maybe_uuid(device_maybe_uuid: Union[int, str]) -> int:
    if isinstance(device_maybe_uuid, int):
        return device_maybe_uuid

    if isinstance(device_maybe_uuid, str):
        for device in range(torch.cuda.device_count()):
            if str(torch.cuda.get_device_properties(device).uuid) == device_maybe_uuid:
                return device
        raise Exception("Invalid device_uuid=" + device_maybe_uuid)

    raise Exception(f"Unknown type: {device_maybe_uuid=}")


def _modify_tuple(t, index: int, modifier: Callable):
    return *t[:index], modifier(t[index]), *t[index + 1 :]


def monkey_patch_torch_compile():
    if torch_release < (2, 8):
        # These things are cacheable by torch.compile. torch.compile just doesn't know it.
        # This was fixed in PyTorch 2.8, but until then, we monkey patch.
        import torch._higher_order_ops.auto_functionalize as af

        af.auto_functionalized_v2._cacheable = True
        af.auto_functionalized._cacheable = True


def register_fake_if_exists(op_name):
    """

View on GitHub (pinned to 0132848349)

Solutions

1. Coerce the value to str or torch.device before passing it downstream
2. Default None device values to a concrete device ('cuda', 'cuda:0') at config load
3. Add a type check/assert where the device value originates

Example fix

# before
set_device(cfg.device)  # cfg.device is None
# after
set_device(cfg.device or 'cuda:0')

assert device_spec is None or isinstance(device_spec, (str, int)), f'bad device {device_spec!r}'

def is_resolvable_device(v) -> bool:
    return v is None or isinstance(v, (str, int))

try:
    _device_from_maybe_uuid(v)
except Exception as e:
    if 'Unknown type' in str(e):
        v = str(v); retry()

When it happens

Trigger: Passing None, a bytes object, a custom enum, or any non-str/non-device type into a device argument that flows through the patched torch API (_device_from_maybe_uuid).

Common situations: Config parsing that yields None instead of a default device, or serialization layers turning UUID strings into bytes.

Related errors

• Invalid device_uuid=
• indices must be on q's device {device}, got {indices.device}
• q, k, and v must be CUDA tensors
• NCCL only supports CUDA, ROCm and MUSA backends.
• Decode context parallel (decode_context_parallel_size > 1) i

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/defdbf6f4a9f7a7c. Report an issue: GitHub.