sgl-project/sglang · error · ValueError

Unknown device module: {device}

Error message

Unknown device module: {device}

What it means

device_context yields a context manager for the correct torch device module. After torch.get_device_module(device) returns None for an unrecognized device string, it raises ValueError('Unknown device module').

Source

Thrown at python/sglang/srt/utils/common.py:254

def get_cuda_version():
    if torch.version.cuda:
        return tuple(map(int, torch.version.cuda.split(".")))
    return (0, 0)


@contextmanager
def device_context(device: torch.device):
    if device.type == "cpu" and is_cpu():
        with torch.device("cpu"):
            yield
    else:
        module = torch.get_device_module(device)
        if module is not None:
            with module.device(device.index):
                yield
        else:
            raise ValueError(f"Unknown device module: {device}")


def _check_cuda_device_version(
    device_capability_majors: List[int], cuda_version: Tuple[int, int]
):
    if not is_cuda():
        return False
    return (
        torch.cuda.get_device_capability()[0] in device_capability_majors
        and tuple(map(int, torch.version.cuda.split(".")[:2])) >= cuda_version
    )


is_ampere_with_cuda_12_3 = lru_cache(maxsize=1)(
    partial(
        _check_cuda_device_version, device_capability_majors=[8], cuda_version=(12, 3)
    )
)

View on GitHub (pinned to 0132848349)

Solutions

1. Normalize/validate the device string before calling (torch.device('cuda:0'), not 'cude:0')
2. For OOT platforms, ensure the platform plugin patches torch device modules before use
3. Pass an explicit supported device ('cuda', 'cpu', 'npu', 'mps', 'musa')

Example fix

# before
with device_context('cude:0'):
# after
with device_context(torch.device('cuda:0')):

dev = torch.device(device_str)  # raises early on malformed strings
assert torch.get_device_module(dev) is not None, f'no device module for {dev}'

def has_device_module(device) -> bool:
    try:
        return torch.get_device_module(device) is not None
    except Exception:
        return False

When it happens

Trigger: Passing a device string torch doesn't map to a module (e.g. 'npu' without torch_npu, a typo like 'cude:0', or an OOT platform string) into device_context, reached e.g. via _layer_norm_fwd.

Common situations: Custom/OOT platforms whose device module isn't registered with torch; malformed device strings from configs; version drift where get_device_module returns None instead of raising.

Related errors

• kv-canary: {name} must be on {reference_name}'s device {refe
• auxiliary PP output {name!r} is not a tensor
• received a non-tensor auxiliary PP output
• The MLX tensor bridge supports CPU and MPS tensors, got {ten
• MlxTensorView requires a Torch MPS tensor, got {owner.device

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/72d058fd3be5eb93. Report an issue: GitHub.