sgl-project/sglang · error · ValueError

Unsupported device type: {device!r}. If this is an OOT platf

Error message

Unsupported device type: {device!r}. If this is an OOT platform, ensure it is properly registered via the 'sglang.platform_plugins' entry point.

What it means

get_available_gpu_memory enumerates known device backends (cuda, npu, mps, musa, ...). For an unrecognized device string it only proceeds if current_platform.is_out_of_tree(); otherwise it raises ValueError telling you the OOT platform must be registered via the 'sglang.platform_plugins' entry point.

Source

Thrown at python/sglang/srt/utils/common.py:525

                "memory allocation for torch MUSA context.",
                gpu_id,
                torch.musa.current_device(),
            )
        if empty_cache:
            empty_device_cache(torch.musa)
        props = torch.musa.get_device_properties(gpu_id)
        if props.is_integrated:
            # On these devices, which use sysmem as device mem, torch.musa.mem_get_info()
            # only reports "free" memory, which can be lower than what is actually
            # available due to not including cache memory. So we use the system available
            # memory metric instead.
            free_gpu_memory = psutil.virtual_memory().available
        free_gpu_memory, total_gpu_memory = torch.musa.mem_get_info()
    elif device == "mps":
        free_gpu_memory = psutil.virtual_memory().available
    else:
        if not current_platform.is_out_of_tree():
            raise ValueError(
                f"Unsupported device type: {device!r}. "
                "If this is an OOT platform, ensure it is properly registered "
                "via the 'sglang.platform_plugins' entry point."
            )
        total_mem = current_platform.get_device_total_memory(gpu_id)
        used_mem = current_platform.get_current_memory_usage()
        free_gpu_memory = total_mem - used_mem

    if distributed:
        tensor = torch.tensor(free_gpu_memory, dtype=torch.float32)
        torch.distributed.all_reduce(
            tensor, op=torch.distributed.ReduceOp.MIN, group=cpu_group
        )
        free_gpu_memory = tensor.item()

    return free_gpu_memory / (1 << 30)

View on GitHub (pinned to 0132848349)

Solutions

1. Install/register your platform via the 'sglang.platform_plugins' entry point implementing is_out_of_tree(), get_device_total_memory, get_current_memory_usage
2. Verify the device string matches a supported backend (cuda/npu/mps/musa/rocm)
3. Update SGLang if your backend is newly supported upstream

Example fix

# before
# custom backend 'xyz' not registered -> ValueError in get_available_gpu_memory
# after
# pyproject.toml of your platform package:
[project.entry-points.'sglang.platform_plugins']
xyz = 'sglang_xyz:register_platform'

supported = {'cuda','npu','mps','musa'}
assert device in supported or current_platform.is_out_of_tree(), device
mem = get_available_gpu_memory(device, gpu_id)

def is_supported_device(device: str) -> bool:
    return device in {'cuda','npu','mps','musa'} or current_platform.is_out_of_tree()

When it happens

Trigger: Calling get_available_gpu_memory with a device string not in the handled list on a non-OOT platform, e.g. a new accelerator or a typo, reached during kernel load (_on_kernel_load).

Common situations: Porting SGLang to a new backend without writing a platform plugin; partial OOT setup where the plugin isn't installed/registered; wrong device id string from env config.

Related errors

• Triton sparse_attn_v4_paged_prefill requires CUDA/HIP tensor
• Block sparse tensors{context} require BLOCK_SIZE_KV={base_n_
• Invalid arch format: {arch_str}
• q, k, and v must be CUDA tensors
• q, k, and v must have the same device and dtype

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c06acf5af1c41080. Report an issue: GitHub.