sgl-project/sglang · error · ValueError

unknown gpu '{gpu}', expected one of {sorted(GPU_BUDGETS_BYT

Error message

unknown gpu '{gpu}', expected one of {sorted(GPU_BUDGETS_BYTES)}

What it means

gpu_budget_bytes looks up per-block shared-memory capacity by GPU name and only knows the keys in GPU_BUDGETS_BYTES (e.g. 'h100'). An unknown GPU string raises ValueError. This is a lookup-table contract for the fused IPM kernel's shmem budget check.

Source

Thrown at python/sglang/kernels/ops/lplb/shmem_budget.py:98

    """Per-array byte breakdown — useful for debugging shmem pressure."""
    b = bytes_per_elem
    return ShmemBreakdown(
        nc=nc,
        nv=nv,
        a_bytes=b * nc * nv,
        c_bytes=b * nv,
        x_bytes=b * nv,
        ata_bytes=b * nc * nc,
        rhs_bytes=b * nc,
        d_bytes=b * nv,
        pad_bytes=_RUNTIME_PAD_BYTES,
    )


def gpu_budget_bytes(gpu: str) -> int:
    key = gpu.lower()
    if key not in GPU_BUDGETS_BYTES:
        raise ValueError(
            f"unknown gpu '{gpu}', expected one of {sorted(GPU_BUDGETS_BYTES)}"
        )
    return GPU_BUDGETS_BYTES[key]


def fits(nc: int, nv: int, gpu: str = "h100") -> bool:
    return shmem_bytes(nc, nv) <= gpu_budget_bytes(gpu)


def assert_fits(nc: int, nv: int, gpu: str = "h100") -> None:
    """Raise if the fused kernel will not fit on the target GPU."""
    used = shmem_bytes(nc, nv)
    cap = gpu_budget_bytes(gpu)
    if used > cap:
        raise ValueError(
            f"fused IPM kernel needs {used/1024:.1f} KiB of shared memory for "
            f"NC={nc}, NV={nv}, but {gpu} allows {cap/1024:.1f} KiB/block. "
            f"Either reduce problem size or switch to a tiled design."

View on GitHub (pinned to 0132848349)

Solutions

1. Pass one of the exact keys printed in the message, e.g. 'h100'
2. If your GPU is legitimately supported hardware, add its budget (bytes per block) to GPU_BUDGETS_BYTES in shmem_budget.py and re-run
3. If the GPU is unsupported, use the non-fused torch reference solver (solve_ipm_torch_reference) instead

Example fix

// before
assert_fits(nc, nv, gpu=torch.cuda.get_device_name(0))

// after
assert_fits(nc, nv, gpu='h100')

from sglang.kernels.ops.lplb.shmem_budget import GPU_BUDGETS_BYTES
gpu = gpu if gpu.lower() in GPU_BUDGETS_BYTES else 'h100'

def is_known_gpu(gpu: str) -> bool:
    return gpu.lower() in GPU_BUDGETS_BYTES

When it happens

Trigger: Calling gpu_budget_bytes('a100'), fits(..., gpu='A100'), assert_fits(..., gpu='l40s') or any solve/warmup path that forwards a GPU name not in the table; also case variants are fine (lowercased) but typos or unlisted GPUs are not.

Common situations: Running on a GPU generation not yet added to the table (e.g. B200); passing a torch device name like 'cuda:0' or 'NVIDIA H100 80GB HBM3' instead of the short key; version skew where the table lags new hardware.

Related errors

• fused IPM kernel needs {used/1024:.1f} KiB of shared memory
• sparse_attn_v4_paged_decode expects fp16/bf16 q, got {q.dtyp
• (head_dim, head_dim_v)=({head_dim}, {head_dim_v}) exceeds SM
• The requested FlashAttention forward configuration exceeds S
• The requested SM120 sheared-bias specialization exceeds shar

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f98dfd7598d0126d. Report an issue: GitHub.