sgl-project/sglang · error · ValueError

Interleaved FC1 N must be even, got {n}

Error message

Interleaved FC1 N must be even, got {n}

What it means

After SwiGLU the kernel halves the output N; because linear and gate rows are interleaved in pairs, the interleaved FC1 N dimension must be even. An odd N means the interleave layout is malformed and output scale-factor indexing would break.

Source

Thrown at python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py:2880

            "nvfp4_gemm_swiglu_nvfp4_quant currently supports NVFP4 input "
            "and output only"
        )
    if a.device.type != "cuda" or b.device.type != "cuda":
        raise ValueError("nvfp4_gemm_swiglu_nvfp4_quant requires CUDA tensors")

    major, minor = get_compute_capability(a.device)
    if major != 10:
        raise ValueError(
            f"nvfp4_gemm_swiglu_nvfp4_quant requires SM100, got SM{major}{minor}"
        )

    m = a.shape[0]
    k = a.shape[1] * 2
    n = b.shape[0]
    if b.shape[1] * 2 != k:
        raise ValueError(f"Shape mismatch: A K={k}, B K={b.shape[1] * 2}")
    if n % 2 != 0:
        raise ValueError(f"Interleaved FC1 N must be even, got {n}")

    l = 1
    n_out = n // 2
    if n_out % sf_vec_size != 0:
        raise ValueError(
            f"Output N={n_out} must be divisible by sf_vec_size={sf_vec_size}"
        )
    scale_n_out = n_out // sf_vec_size
    padded_m = _round_up(m, 128)
    padded_scale_n = _round_up(scale_n_out, 4)

    ab_dtype_cutlass = get_cutlass_dtype(ab_dtype)
    sf_dtype_cutlass = get_cutlass_dtype(sf_dtype)
    c_dtype_cutlass = get_cutlass_dtype(c_dtype)

    if m <= 128:
        mma_tiler_mn, cluster_shape_mn = (128, 128), (1, 2)
    else:

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure FC1 weights go through interleave_linear_and_gate before this op
2. Verify b.shape[0] == 2*intermediate_size (even by construction)

Example fix

# before
b = torch.cat([linear_w, gate_w], dim=0)
out = nvfp4_gemm_swiglu_nvfp4_quant(a, b, ...)
# after
b = interleave_linear_and_gate(torch.cat([linear_w, gate_w], dim=0), group_size)
out = nvfp4_gemm_swiglu_nvfp4_quant(a, b, ...)

assert b.shape[0] % 2 == 0

def interleaved_n_ok(b): return b.shape[0] % 2 == 0

When it happens

Trigger: Calling nvfp4_gemm_swiglu_nvfp4_quant with b.shape[0] (interleaved N) odd — typically because interleave_linear_and_gate was skipped or applied with wrong group_size, leaving a concatenated rather than interleaved layout.

Common situations: Forgetting to run interleave_linear_and_gate on FC1 weights during process_weights_after_loading, or hand-building b for a test with odd row count.

Related errors

• expected a tensor with at least one dimension
• dimension {dim} size {dim_size} must be divisible by 2 * gro
• The pointers must be multiple of 16 bytes.
• The last dimension ({input.shape[-1]}) x itemsize ({input.dt
• rope_pool_fused expects q/k/v to be 3-D

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/76f2c3fcf87b9ee2. Report an issue: GitHub.