{"record":{"id":"680eb76dff0d5613","repo":"sgl-project/sglang","slug":"output-n-n-out-must-be-divisible-by-sf-vec-size","errorCode":null,"errorMessage":"Output N={n_out} must be divisible by sf_vec_size={sf_vec_size}","messagePattern":"Output N=(.+?) must be divisible by sf_vec_size=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py","lineNumber":2885,"sourceCode":"\n    major, minor = get_compute_capability(a.device)\n    if major != 10:\n        raise ValueError(\n            f\"nvfp4_gemm_swiglu_nvfp4_quant requires SM100, got SM{major}{minor}\"\n        )\n\n    m = a.shape[0]\n    k = a.shape[1] * 2\n    n = b.shape[0]\n    if b.shape[1] * 2 != k:\n        raise ValueError(f\"Shape mismatch: A K={k}, B K={b.shape[1] * 2}\")\n    if n % 2 != 0:\n        raise ValueError(f\"Interleaved FC1 N must be even, got {n}\")\n\n    l = 1\n    n_out = n // 2\n    if n_out % sf_vec_size != 0:\n        raise ValueError(\n            f\"Output N={n_out} must be divisible by sf_vec_size={sf_vec_size}\"\n        )\n    scale_n_out = n_out // sf_vec_size\n    padded_m = _round_up(m, 128)\n    padded_scale_n = _round_up(scale_n_out, 4)\n\n    ab_dtype_cutlass = get_cutlass_dtype(ab_dtype)\n    sf_dtype_cutlass = get_cutlass_dtype(sf_dtype)\n    c_dtype_cutlass = get_cutlass_dtype(c_dtype)\n\n    if m <= 128:\n        mma_tiler_mn, cluster_shape_mn = (128, 128), (1, 2)\n    else:\n        mma_tiler_mn, cluster_shape_mn = (256, 128), (2, 1)\n\n    if not Sm100BlockScaledPersistentDenseGemmKernel.can_implement(\n        ab_dtype_cutlass,\n        sf_dtype_cutlass,","sourceCodeStart":2867,"sourceCodeEnd":2903,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py#L2867-L2903","documentation":"The output is re-quantized to NVFP4 in groups of sf_vec_size (16) elements, so the post-SwiGLU output width n/2 must be divisible by sf_vec_size for the scale-factor tensor to have an integer number of groups. Otherwise the output scale layout would be fractional/unaligned.","triggerScenarios":"Calling nvfp4_gemm_swiglu_nvfp4_quant where (b.shape[0]//2) % sf_vec_size != 0, e.g. intermediate_size not a multiple of 16.","commonSituations":"Custom models with unusual intermediate sizes, or overriding sf_vec_size to a nonstandard value while the model geometry stays fixed.","solutions":["Ensure intermediate_size is a multiple of sf_vec_size (16)","Keep sf_vec_size at the default 16 expected by NVFP4 kernels","Fall back to a non-fused path for models with unaligned intermediate size"],"exampleFix":"# before\nsf_vec_size = 16  # intermediate_size = 5000 -> n_out=2500 not divisible\n# after\n# use intermediate_size aligned to 16, e.g. 4992 or 5016","handlingStrategy":"validation","validationCode":"assert (b.shape[0] // 2) % sf_vec_size == 0","typeGuard":"def n_out_aligned(b, sf=16): return (b.shape[0]//2) % sf == 0","tryCatchPattern":null,"preventionTips":["Keep sf_vec_size at the NVFP4 default of 16","Validate intermediate_size alignment at config load"],"tags":["nvfp4","shape-alignment","scale-factor","quantization"],"backgroundTag":"tensor-shape-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}