sgl-project/sglang · error · RuntimeError

LPLB fused solver requires CUDA tensors; got A on {A.device}

Error message

LPLB fused solver requires CUDA tensors; got A on {A.device}.

What it means

solve_ipm requires all inputs on CUDA; passing A on CPU (or any non-CUDA device) raises immediately. The fused kernel launches on the tensor's device with no host fallback.

Source

Thrown at python/sglang/kernels/ops/lplb/torch_solver.py:129

    Args:
        A: Constraint matrix, shape (NC, NV), float32, on CUDA.
        b: RHS vector, shape (NC,), float32, on CUDA.
        c: Objective coefficients, shape (NV,), float32, on CUDA.
        num_iters: Number of barrier iterations (default 5).

    Returns:
        x: Solution vector, shape (NV,), float32. The kernel writes 0.5
        for every entry on non-convergence.
    """
    nc, nv = A.shape
    assert b.shape == (nc,), f"b shape mismatch: {b.shape} vs ({nc},)"
    assert c.shape == (nv,), f"c shape mismatch: {c.shape} vs ({nv},)"

    _init_fused_backend()
    if not _FUSED_AVAILABLE:
        raise RuntimeError(f"LPLB fused solver unavailable: {_unavailable_reason()}")
    if not A.is_cuda:
        raise RuntimeError(
            f"LPLB fused solver requires CUDA tensors; got A on {A.device}."
        )
    if A.dtype != torch.float32:
        raise RuntimeError(
            f"LPLB fused solver requires float32; got A.dtype={A.dtype}."
        )
    return _FUSED_SOLVE_IPM(A, b, c, num_iters=num_iters)


def solve_ipm_torch_reference(
    A: torch.Tensor,
    b: torch.Tensor,
    c: torch.Tensor,
    num_iters: int = 5,
) -> torch.Tensor:
    """Pure-torch reference for the fused IPM kernel — testing only.

    Mirrors the barrier-method iteration in ``csrc/lplb/ipm.cuh``

View on GitHub (pinned to 0132848349)

Solutions

1. Move all inputs to the same CUDA device: A, b, c = A.cuda(), b.cuda(), c.cuda() (or .to(device))
2. Add an assert A.is_cuda before calling solve_ipm in your pipeline
3. Use solve_ipm_torch_reference for CPU-side debugging only

Example fix

// before
x, y, s = solve_ipm(A, b, c)  # A on cpu

// after
device = 'cuda'
x, y, s = solve_ipm(A.to(device), b.to(device), c.to(device))

device = dst_device if dst_device is not None else 'cuda'
A, b, c = A.to(device), b.to(device), c.to(device)
assert A.is_cuda

def all_cuda(*ts: torch.Tensor) -> bool:
    return all(t.is_cuda for t in ts)

When it happens

Trigger: Calling solve_ipm(A, b, c) with A created via torch.randn(...) without .cuda(), or tensors left on 'cpu' after a checkpoint load; the check only inspects A.

Common situations: Prototype scripts that forgot .to('cuda'); models moved to GPU except one operand; b or c on CPU with A on GPU passes this check but may fail inside the kernel — always move all three.

Related errors

• {name}_block_cnt and {name}_block_idx must be on the same de
• indices must be on q's device {device}, got {indices.device}
• topk_length must be a CUDA tensor
• topk_length must be on q's device {device}, got {topk_length
• QKV and cos/sin tensors must be on the same CUDA device

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/99806a253aadfae6. Report an issue: GitHub.