{"record":{"id":"cb91e183b637c9cb","repo":"sgl-project/sglang","slug":"the-pointers-must-be-multiple-of-16-bytes","errorCode":null,"errorMessage":"The pointers must be multiple of 16 bytes.","messagePattern":"The pointers must be multiple of 16 bytes\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/aot/python/sgl_kernel/elementwise.py","lineNumber":260,"sourceCode":"            input, residual, weight, eps, enable_pdl\n        )\n    else:\n        _gemma_fused_add_rmsnorm_internal(input, residual, weight, eps, enable_pdl)\n\n\ndef _check_shape(input: torch.Tensor, output: torch.Tensor) -> None:\n    assert input.ndim == output.ndim, f\"{input.ndim} != {output.ndim}\"\n    assert (\n        input.shape[:-1] == output.shape[:-1]\n    ), f\"{input.shape[:-1]} != {output.shape[:-1]}\"\n    assert (\n        input.shape[-1] == 2 * output.shape[-1]\n    ), f\"{input.shape[-1]} != {2 * output.shape[-1]}\"\n\n\ndef silu_and_mul(input: torch.Tensor, out: torch.Tensor = None) -> torch.Tensor:\n    if input.shape[-1] * input.dtype.itemsize % 16 != 0:\n        raise ValueError(\"The pointers must be multiple of 16 bytes.\")\n    if out is not None:\n        _check_shape(input, out)\n    else:\n        out = torch.empty(\n            input.shape[:-1] + (input.shape[-1] // 2,),\n            device=input.device,\n            dtype=input.dtype,\n        )\n    torch.ops.sgl_kernel.silu_and_mul.default(out, input)\n    return out\n\n\ndef gelu_tanh_and_mul(input: torch.Tensor, out: torch.Tensor = None) -> torch.Tensor:\n    if input.shape[-1] * input.dtype.itemsize % 16 != 0:\n        raise ValueError(\"The pointers must be multiple of 16 bytes.\")\n    if out is not None:\n        _check_shape(input, out)\n    else:","sourceCodeStart":242,"sourceCodeEnd":278,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/aot/python/sgl_kernel/elementwise.py#L242-L278","documentation":"silu_and_mul validates that input.shape[-1] * input.dtype.itemsize is a multiple of 16 bytes because the CUDA kernel uses 128-bit vectorized loads. The message says 'pointers' but the actual check is on the last-dimension byte size; e.g. bf16 input with an odd hidden size (2*h bytes not divisible by 16) fails.","triggerScenarios":"Calling sgl_kernel.elementwise.silu_and_mul with hidden dim * itemsize % 16 != 0, e.g. fp32 with d%4!=0 or bf16 with d%8!=0; using a custom model whose intermediate size after the concat is not 16-byte aligned in bytes.","commonSituations":"Porting a model with an unusual MLP intermediate size; testing kernels with toy tensors of tiny/odd sizes.","solutions":["Choose a hidden size whose byte length is 16-byte aligned (bf16: multiple of 8 elements; fp16: multiple of 8; fp32: multiple of 4).","Pad the last dimension to alignment before the call and slice afterwards if exactness matters.","Use the PyTorch-native fallback (torch.nn.functional.silu) for non-aligned shapes."],"exampleFix":"# before\nout = silu_and_mul(x)  # x: bf16, shape[-1] = 5000 (10000 bytes, not %16)\n# after\nx = torch.nn.functional.pad(x, (0, 8))\nout = silu_and_mul(x)[..., :2500]","handlingStrategy":"validation","validationCode":"def silu_mul_ready(x: torch.Tensor) -> bool:\n    return x.shape[-1] * x.dtype.itemsize % 16 == 0\nassert silu_mul_ready(x)","typeGuard":"def silu_mul_ready(x: torch.Tensor) -> bool:\n    return x.dim() > 0 and x.shape[-1] * x.dtype.itemsize % 16 == 0","tryCatchPattern":"try:\n    out = silu_and_mul(x)\nexcept ValueError:\n    out = torch.nn.functional.silu(x[..., :d]) * x[..., d:]  # fallback","preventionTips":["Keep MLP hidden sizes multiples of 8 (bf16/fp16) or 4 (fp32).","In model ports, assert byte-alignment of fused activation inputs before serving."],"tags":["sglang","cuda-kernel","alignment","silu","shape-validation"],"backgroundTag":"vectorized-kernel-alignment","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}