sgl-project/sglang · error · ValueError

dimension {dim} size {dim_size} must be divisible by 2 * gro

Error message

dimension {dim} size {dim_size} must be divisible by 2 * group_size={2 * group_size}

What it means

The interleaving groups linear/gate rows into chunks of group_size pairs, so the dimension being interleaved must be divisible by 2*group_size (a linear chunk plus a gate chunk). Any remainder makes the layout ambiguous for the fused kernel, hence the hard error.

Source

Thrown at python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py:2713

def interleave_linear_and_gate(
    tensor: torch.Tensor,
    group_size: int = 64,
    dim: int = 0,
) -> torch.Tensor:
    """Rewrite ``[linear all][gate all]`` along ``dim`` as
    ``[linear chunk][gate chunk]…`` with ``group_size`` rows per chunk.

    Matches the FC1 GEMM+SwiGLU layout the fused-gemm kernel expects.
    """
    if tensor.ndim == 0:
        raise ValueError("expected a tensor with at least one dimension")
    dim = dim % tensor.ndim
    sizes = tensor.size()
    dim_size = sizes[dim]
    if dim_size % (group_size * 2) != 0:
        raise ValueError(
            f"dimension {dim} size {dim_size} must be divisible by "
            f"2 * group_size={2 * group_size}"
        )
    prev_sizes = sizes[:dim]
    post_sizes = sizes[dim + 1 :]
    return (
        tensor.reshape(
            *prev_sizes,
            2,
            dim_size // (group_size * 2),
            group_size,
            *post_sizes,
        )
        .transpose(dim, dim + 1)
        .reshape(*sizes)
        .contiguous()
    )

View on GitHub (pinned to 0132848349)

Solutions

1. Check intermediate_size % group_size == 0 for the model; use a matching group_size
2. Interleave along the correct dim (typically dim=0 of the stacked weight)
3. If the model geometry is inherently unaligned, fall back to a non-fused SwiGLU path

Example fix

// before
w = interleave_linear_and_gate(fc1_w, group_size=128)  # dim0 = 2*5000
// after
# use aligned geometry: intermediate_size multiple of group_size
w = interleave_linear_and_gate(fc1_w_aligned, group_size=128)

assert tensor.size(dim) % (2*group_size) == 0, (tensor.size(dim), group_size)

def fc1_interleavable(w, gs, dim=0): return w.size(dim) % (2*gs) == 0

When it happens

Trigger: Calling interleave_linear_and_gate on an FC1 weight whose concatenated dim (2*intermediate_size) is not divisible by 2*group_size, e.g. intermediate_size not a multiple of group_size (commonly 128).

Common situations: Running a model whose intermediate_size is not aligned to the kernel's group size, or passing the wrong group_size (e.g. 64 vs 128) during process_weights_after_loading; misconfigured custom architectures.

Related errors

• expected a tensor with at least one dimension
• Interleaved FC1 N must be even, got {n}
• Output N={n_out} must be divisible by sf_vec_size={sf_vec_si
• Unsupported nvfp4_gemm_swiglu_nvfp4_quant configuration: sha
• unsupported input for packed fused SiLU-mul

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/87c4a1441d07f7bf. Report an issue: GitHub.