sgl-project/sglang · error · ValueError

expected a tensor with at least one dimension

Error message

expected a tensor with at least one dimension

What it means

interleave_linear_and_gate rewrites a concatenated [linear weights; gate weights] matrix into interleaved chunks for the fused FC1 GEMM+SwiGLU kernel. It requires at least a 1-D tensor; a 0-dim (scalar) tensor has no dimension to interleave along, so it is rejected immediately.

Source

Thrown at python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py:2708

from flashinfer.utils import get_compute_capability  # noqa: E402


def _round_up(value: int, multiple: int) -> int:
    return (value + multiple - 1) // multiple * multiple


def interleave_linear_and_gate(
    tensor: torch.Tensor,
    group_size: int = 64,
    dim: int = 0,
) -> torch.Tensor:
    """Rewrite ``[linear all][gate all]`` along ``dim`` as
    ``[linear chunk][gate chunk]…`` with ``group_size`` rows per chunk.

    Matches the FC1 GEMM+SwiGLU layout the fused-gemm kernel expects.
    """
    if tensor.ndim == 0:
        raise ValueError("expected a tensor with at least one dimension")
    dim = dim % tensor.ndim
    sizes = tensor.size()
    dim_size = sizes[dim]
    if dim_size % (group_size * 2) != 0:
        raise ValueError(
            f"dimension {dim} size {dim_size} must be divisible by "
            f"2 * group_size={2 * group_size}"
        )
    prev_sizes = sizes[:dim]
    post_sizes = sizes[dim + 1 :]
    return (
        tensor.reshape(
            *prev_sizes,
            2,
            dim_size // (group_size * 2),
            group_size,
            *post_sizes,
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect the tensor's shape before calling; restore the expected 2-D [2*intermediate, hidden] weight
2. Fix upstream .squeeze()/.item() calls that collapsed the weight to 0-dim

Example fix

// before
t = torch.tensor(1.0)
w = interleave_linear_and_gate(t, group_size=64)
// after
t = t.reshape(1, 1)
w = interleave_linear_and_gate(t, group_size=64)

if tensor.ndim == 0: raise ValueError(f'weight must be >=1-D, got shape {tuple(tensor.shape)}')

def is_non_scalar(t): return t.ndim >= 1

When it happens

Trigger: Calling interleave_linear_and_gate on a 0-dimensional tensor (e.g. tensor created with torch.tensor(1.0) or an over-squeezed weight).

Common situations: Weight-loading bugs where an accidental .squeeze()/item() collapses the FC1 weight to a scalar, or unit tests passing dummy scalar tensors.

Related errors

• dimension {dim} size {dim_size} must be divisible by 2 * gro
• Interleaved FC1 N must be even, got {n}
• Invalid gate_up_proj shape for {name}: {tuple(loaded_weight.
• The pointers must be multiple of 16 bytes.
• The last dimension ({input.shape[-1]}) x itemsize ({input.dt

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f0c653787a472aef. Report an issue: GitHub.