{"record":{"id":"f2ae37c4ef5322ec","repo":"sgl-project/sglang","slug":"the-last-dimension-input-shape-1-x-itemsize","errorCode":null,"errorMessage":"The last dimension ({input.shape[-1]}) x itemsize ({input.dtype.itemsize}) must be a multiple of 16 bytes.","messagePattern":"The last dimension \\((.+?)\\) x itemsize \\((.+?)\\) must be a multiple of 16 bytes\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/aot/python/sgl_kernel/elementwise.py","lineNumber":313,"sourceCode":"            input.shape[:-1] + (input.shape[-1] // 2,),\n            device=input.device,\n            dtype=input.dtype,\n        )\n    torch.ops.sgl_kernel.gelu_and_mul.default(out, input)\n    return out\n\n\nif torch.version.hip is not None:\n\n    def gelu_quick(input: torch.Tensor, out: torch.Tensor = None) -> torch.Tensor:\n        \"\"\"\n        Quick-GELU:  y = x * sigmoid(1.702 * x)\n\n        The CUDA/HIP kernel uses 128-bit (16-byte) vector loads & stores,\n        so the last-dimension byte length must be a multiple of 16 bytes.\n        \"\"\"\n        if input.shape[-1] * input.dtype.itemsize % 16 != 0:\n            raise ValueError(\n                f\"The last dimension ({input.shape[-1]}) x itemsize \"\n                f\"({input.dtype.itemsize}) must be a multiple of 16 bytes.\"\n            )\n\n        if out is not None:\n            assert input.shape == out.shape, f\"{input.shape} != {out.shape}\"\n        else:\n            out = torch.empty_like(input)\n\n        torch.ops.sgl_kernel.gelu_quick(out, input)\n        return out\n\n\ndef dsv4_fused_q_norm_rope(\n    q_input: torch.Tensor,\n    freqs_cis: torch.Tensor,\n    positions: torch.Tensor,\n    eps: float = 1e-6,","sourceCodeStart":295,"sourceCodeEnd":331,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/aot/python/sgl_kernel/elementwise.py#L295-L331","documentation":"gelu_quick (QuickGELU, y = x*sigmoid(1.702x)) requires the last dimension in bytes to be a multiple of 16 because the CUDA/HIP kernel does 128-bit vectorized loads/stores. Unlike the *_and_mul variants it reports the dimension, itemsize, and computed requirement explicitly.","triggerScenarios":"Calling gelu_quick with last-dim * itemsize % 16 != 0 — e.g. fp32 tensor of width 7, bf16 width 12; Qwen-style QuickGELU MLPs with non-aligned sizes (Qwen sizes are normally aligned).","commonSituations":"Testing the kernel with tiny toy tensors; custom architectures with odd feature widths.","solutions":["Pick a feature width aligned to 16 bytes (bf16/fp16: %8==0, fp32: %4==0).","Pad last dim to alignment and slice after: gelu_quick(pad(x))[..., :d].","Fall back to x * torch.sigmoid(1.702*x)."],"exampleFix":"# before\ny = gelu_quick(x)  # x fp32, last dim 7\n# after\ny = gelu_quick(torch.nn.functional.pad(x, (0,1)))[..., :7]","handlingStrategy":"validation","validationCode":"if input.shape[-1] * input.dtype.itemsize % 16 != 0:\n    input = pad_to_16b(input)","typeGuard":"def gelu_quick_ready(x: torch.Tensor) -> bool:\n    return x.shape[-1] * x.dtype.itemsize % 16 == 0","tryCatchPattern":"try:\n    y = gelu_quick(x)\nexcept ValueError:\n    y = x * torch.sigmoid(1.702 * x)","preventionTips":["Choose feature widths divisible by 8 (half precision) / 4 (fp32).","Wrap the call in a helper that pads and slices so callers never see the error."],"tags":["sglang","cuda-kernel","alignment","quick-gelu","shape-validation"],"backgroundTag":"vectorized-kernel-alignment","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}