sgl-project/sglang · error · ValueError

fused IPM kernel needs {used/1024:.1f} KiB of shared memory

Error message

fused IPM kernel needs {used/1024:.1f} KiB of shared memory for NC={nc}, NV={nv}, but {gpu} allows {cap/1024:.1f} KiB/block. Either reduce problem size or switch to a tiled design.

What it means

assert_fits computes the shared memory required by the fused interior-point-method kernel for the given NC/NV (constraint/variable counts) and compares it against the per-block shmem cap of the target GPU. Exceeding the cap means the kernel cannot launch. The error is a capacity-planning guard, not a runtime CUDA failure.

Source

Thrown at python/sglang/kernels/ops/lplb/shmem_budget.py:113

def gpu_budget_bytes(gpu: str) -> int:
    key = gpu.lower()
    if key not in GPU_BUDGETS_BYTES:
        raise ValueError(
            f"unknown gpu '{gpu}', expected one of {sorted(GPU_BUDGETS_BYTES)}"
        )
    return GPU_BUDGETS_BYTES[key]


def fits(nc: int, nv: int, gpu: str = "h100") -> bool:
    return shmem_bytes(nc, nv) <= gpu_budget_bytes(gpu)


def assert_fits(nc: int, nv: int, gpu: str = "h100") -> None:
    """Raise if the fused kernel will not fit on the target GPU."""
    used = shmem_bytes(nc, nv)
    cap = gpu_budget_bytes(gpu)
    if used > cap:
        raise ValueError(
            f"fused IPM kernel needs {used/1024:.1f} KiB of shared memory for "
            f"NC={nc}, NV={nv}, but {gpu} allows {cap/1024:.1f} KiB/block. "
            f"Either reduce problem size or switch to a tiled design."
        )


def max_nc_for_nv(nv: int, gpu: str = "h100") -> int:
    """Largest NC that fits for a given NV. Solves
        4 * (NC^2 + (NV+1)*NC + 3*NV) + pad <= cap
    via the quadratic formula (monotone in NC). Returns 0 if even NC=1 overflows.
    """
    cap = gpu_budget_bytes(gpu)
    b = _BYTES_PER_ELEM
    # cap - pad >= b * (NC^2 + (NV+1)*NC + 3*NV)
    rhs = (cap - _RUNTIME_PAD_BYTES) / b - 3 * nv
    if rhs <= 0:
        return 0
    # NC^2 + (NV+1)*NC - rhs <= 0

View on GitHub (pinned to 0132848349)

Solutions

  1. Reduce nc or nv below the budget (the message states exact required vs allowed KiB)
  2. Switch to the tiled / torch reference path (solve_ipm_torch_reference) which does not use one-block shared memory
  3. Target a GPU with a larger per-block shmem budget (e.g. h100) if available

Example fix

// before
warmup(nc=8192, nv=8192)  # exceeds H100 227 KiB/block

// after
from sglang.kernels.ops.lplb.torch_solver import solve_ipm_torch_reference
x, y, s = solve_ipm_torch_reference(A, b, c)
Defensive patterns

Strategy: validation

Validate before calling

from sglang.kernels.ops.lplb.shmem_budget import fits
if not fits(nc, nv, gpu='h100'):
    use_fused = False  # route to reference solver

Try / catch

try:
    assert_fits(nc, nv, gpu)
except ValueError as e:
    logger.warning('%s; falling back to torch reference', e)
    result = solve_ipm_torch_reference(A, b, c)

Prevention

When it happens

Trigger: Calling assert_fits(nc, nv, gpu) or LPLB warmup/solve with problem sizes whose shmem_bytes(nc, nv) exceeds gpu_budget_bytes(gpu); larger NC*NV combos on smaller-shmem GPUs (e.g. A100's 164 KiB vs H100's 228 KiB).

Common situations: Scaling up the LPLB problem (more constraints/vars) until it exceeds ~227 KiB on H100; targeting a consumer GPU with 99/100 KiB caps; forgetting that shmem scales with both nc and nv.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3944a04f0b16c984. Report an issue: GitHub.