{"record":{"id":"87c4a1441d07f7bf","repo":"sgl-project/sglang","slug":"dimension-dim-size-dim-size-must-be-divisible","errorCode":null,"errorMessage":"dimension {dim} size {dim_size} must be divisible by 2 * group_size={2 * group_size}","messagePattern":"dimension (.+?) size (.+?) must be divisible by 2 \\* group_size=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py","lineNumber":2713,"sourceCode":"\n\ndef interleave_linear_and_gate(\n    tensor: torch.Tensor,\n    group_size: int = 64,\n    dim: int = 0,\n) -> torch.Tensor:\n    \"\"\"Rewrite ``[linear all][gate all]`` along ``dim`` as\n    ``[linear chunk][gate chunk]…`` with ``group_size`` rows per chunk.\n\n    Matches the FC1 GEMM+SwiGLU layout the fused-gemm kernel expects.\n    \"\"\"\n    if tensor.ndim == 0:\n        raise ValueError(\"expected a tensor with at least one dimension\")\n    dim = dim % tensor.ndim\n    sizes = tensor.size()\n    dim_size = sizes[dim]\n    if dim_size % (group_size * 2) != 0:\n        raise ValueError(\n            f\"dimension {dim} size {dim_size} must be divisible by \"\n            f\"2 * group_size={2 * group_size}\"\n        )\n    prev_sizes = sizes[:dim]\n    post_sizes = sizes[dim + 1 :]\n    return (\n        tensor.reshape(\n            *prev_sizes,\n            2,\n            dim_size // (group_size * 2),\n            group_size,\n            *post_sizes,\n        )\n        .transpose(dim, dim + 1)\n        .reshape(*sizes)\n        .contiguous()\n    )\n","sourceCodeStart":2695,"sourceCodeEnd":2731,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py#L2695-L2731","documentation":"The interleaving groups linear/gate rows into chunks of group_size pairs, so the dimension being interleaved must be divisible by 2*group_size (a linear chunk plus a gate chunk). Any remainder makes the layout ambiguous for the fused kernel, hence the hard error.","triggerScenarios":"Calling interleave_linear_and_gate on an FC1 weight whose concatenated dim (2*intermediate_size) is not divisible by 2*group_size, e.g. intermediate_size not a multiple of group_size (commonly 128).","commonSituations":"Running a model whose intermediate_size is not aligned to the kernel's group size, or passing the wrong group_size (e.g. 64 vs 128) during process_weights_after_loading; misconfigured custom architectures.","solutions":["Check intermediate_size % group_size == 0 for the model; use a matching group_size","Interleave along the correct dim (typically dim=0 of the stacked weight)","If the model geometry is inherently unaligned, fall back to a non-fused SwiGLU path"],"exampleFix":"// before\nw = interleave_linear_and_gate(fc1_w, group_size=128)  # dim0 = 2*5000\n// after\n# use aligned geometry: intermediate_size multiple of group_size\nw = interleave_linear_and_gate(fc1_w_aligned, group_size=128)","handlingStrategy":"validation","validationCode":"assert tensor.size(dim) % (2*group_size) == 0, (tensor.size(dim), group_size)","typeGuard":"def fc1_interleavable(w, gs, dim=0): return w.size(dim) % (2*gs) == 0","tryCatchPattern":null,"preventionTips":["Validate intermediate_size alignment against group_size at model load","Fail fast in config validation for unaligned custom models"],"tags":["nvfp4","shape-alignment","weight-loading","swiglu"],"backgroundTag":"tensor-shape-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}