sgl-project/sglang · error · ValueError

Unsupported nvfp4_gemm_swiglu_nvfp4_quant configuration: sha

Error message

Unsupported nvfp4_gemm_swiglu_nvfp4_quant configuration: shape=(M={m}, N={n}, K={k}), mma_tiler_mn={mma_tiler_mn}, cluster_shape_mn={cluster_shape_mn}

What it means

Before launching, the op validates the problem shape against the CUTLASS MMA tiler and cluster shape (e.g. requires N/K to be coverable by the chosen tile/cluster geometry on SM100). If the M,N,K combination cannot be tiled by mma_tiler_mn with cluster_shape_mn, the configuration is unsupported and it refuses to launch rather than computing garbage.

Source

Thrown at python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py:2916

    else:
        mma_tiler_mn, cluster_shape_mn = (256, 128), (2, 1)

    if not Sm100BlockScaledPersistentDenseGemmKernel.can_implement(
        ab_dtype_cutlass,
        sf_dtype_cutlass,
        sf_vec_size,
        c_dtype_cutlass,
        mma_tiler_mn,
        cluster_shape_mn,
        m,
        n,
        k,
        l,
        a_major="k",
        b_major="k",
        c_major="n",
    ):
        raise ValueError(
            "Unsupported nvfp4_gemm_swiglu_nvfp4_quant configuration: "
            f"shape=(M={m}, N={n}, K={k}), mma_tiler_mn={mma_tiler_mn}, "
            f"cluster_shape_mn={cluster_shape_mn}"
        )

    if out is None:
        out = torch.empty((m, n_out // 2), dtype=torch.uint8, device=a.device)
    if out_scale is None:
        out_scale = torch.empty(
            (padded_m, padded_scale_n),
            dtype=torch.float8_e4m3fn,
            device=a.device,
        )

    if alpha.dim() == 0 or alpha.dim() == 1:
        alpha = alpha.view(1, 1)
    if output_global_scale.dim() == 0:
        output_global_scale = output_global_scale.view(1)

View on GitHub (pinned to 0132848349)

Solutions

1. Round or pad K and N to multiples required by the tiler (commonly 256 for the tiler_mn used)
2. Use the default mma_tiler_mn/cluster_shape_mn unless you have verified shape compatibility
3. If geometry is fixed and unaligned, use a non-fused GEMM path for that layer

Example fix

# before
out = nvfp4_gemm_swiglu_nvfp4_quant(a, b, sf_a, sf_b, mma_tiler_mn=(256,128), ...)
# after
# pick a tiler whose spans divide K and N, or pad K/N to alignment
out = nvfp4_gemm_swiglu_nvfp4_quant(a_padded, b_padded, ... )

m, k, n = a.shape[0], a.shape[1]*2, b.shape[0]
# check divisibility by your tiler spans, e.g. 256
assert k % 256 == 0 and n % tiler_n == 0, (m, n, k)

def shape_supported(m, n, k, tiler_mn, cluster_mn):
    return n % (tiler_mn[0]*cluster_mn[0]) == 0 and k % (tiler_mn[1]*cluster_mn[1]) == 0

try:
    out = nvfp4_gemm_swiglu_nvfp4_quant(...)
except ValueError:
    out = fallback_gemm_swiglu_quant(...)

When it happens

Trigger: Calling with shape values not divisible by the tile requirements — commonly K or N not a multiple of the tiler span (e.g. 256) implied by mma_tiler_mn and cluster_shape_mn, or extreme shapes like M=0.

Common situations: Custom layer geometries with unaligned hidden/intermediate sizes, overriding mma_tiler_mn/cluster_shape_mn for tuning without matching shape alignment, or very small N/K leftovers after packing.

Related errors

• Unsupported CUTLASS scalar type for A/B: {cutlass_type!r}
• Unsupported CUTLASS scalar type for accumulator: {cutlass_ty
• M must be 64, 128 or 256
• N must be a multiple of 8 in the range 8…256
• Unexpected swizzle shift – want S==3 for M==4

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0985e9e15c951612. Report an issue: GitHub.