{"record":{"id":"0854c8ab40ef2d72","repo":"sgl-project/sglang","slug":"unsupported-dtype-dtype-supported-float16-bfl","errorCode":null,"errorMessage":"Unsupported dtype {dtype}. Supported: float16, bfloat16, float32","messagePattern":"Unsupported dtype (.+?)\\. Supported: float16, bfloat16, float32","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/mamba/causal_conv1d.py","lineNumber":28,"sourceCode":"\nimport torch\n\nfrom sglang.kernels.jit.utils import (\n    cache_once,\n    get_jit_cuda_arch,\n    load_jit,\n    make_cpp_args,\n)\nfrom sglang.srt.utils.custom_op import register_custom_op\n\nif TYPE_CHECKING:\n    from tvm_ffi.module import Module\n\n\n@cache_once\ndef _jit_causal_conv1d_module(dtype: torch.dtype) -> Module:\n    if dtype not in (torch.float16, torch.bfloat16, torch.float32):\n        raise RuntimeError(\n            f\"Unsupported dtype {dtype}. Supported: float16, bfloat16, float32\"\n        )\n    # The AOT wheel ships an SM90 build compiled with `-use_fast_math` and a\n    # precise-math build for every other arch; match that split so the SiLU\n    # epilogue keeps producing the same bits as the op being replaced.\n    arch = get_jit_cuda_arch()\n    use_fast_math = (arch.major, arch.minor) == (9, 0)\n    math_mode = \"fast_math\" if use_fast_math else \"precise_math\"\n    args = make_cpp_args(dtype)\n    return load_jit(\n        \"causal_conv1d\",\n        math_mode,\n        *args,\n        cuda_files=[\"mamba/causal_conv1d.cuh\"],\n        cuda_wrappers=[\n            (\"causal_conv1d_fwd\", f\"causal_conv1d_fwd<{args}>\"),\n            (\"causal_conv1d_update\", f\"causal_conv1d_update<{args}>\"),\n        ],","sourceCodeStart":10,"sourceCodeEnd":46,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/mamba/causal_conv1d.py#L10-L46","documentation":"The causal-conv1d JIT module factory only compiles kernels for float16, bfloat16, and float32. Any other dtype (float64, int, etc.) raises before TVM-FFI module build. Both causal_conv1d_fwd and causal_conv1d_update route through this cached factory, so the error surfaces on first use of a given dtype.","triggerScenarios":"Calling causal_conv1d_fwd or causal_conv1d_update with x/weight in float64 (numpy default), or an integer/other dtype tensor.","commonSituations":"Mamba/conv states created via torch.from_numpy(...) which defaults to float64; test fixtures with wrong dtype; mixed input pipelines after .double() debugging.","solutions":["Cast the conv inputs and states to one of float16/bfloat16/float32 (usually the model dtype, e.g. .to(torch.bfloat16))","Fix the tensor creation site (from_numpy(...).float() or dtype= in torch.randn) rather than casting at call time","Keep hidden states and conv states in the same dtype as the model to avoid downstream mismatches"],"exampleFix":"// before\ny = causal_conv1d_fwd(torch.from_numpy(x_np), weight, ...)\n\n// after\ny = causal_conv1d_fwd(torch.from_numpy(x_np).to(torch.bfloat16), weight, ...)","handlingStrategy":"type-guard","validationCode":"assert x.dtype in (torch.float16, torch.bfloat16, torch.float32), f'bad dtype {x.dtype}'","typeGuard":"def conv_dtype_ok(x: torch.Tensor) -> bool:\n    return x.dtype in (torch.float16, torch.bfloat16, torch.float32)","tryCatchPattern":null,"preventionTips":["Create mamba conv states with the model dtype explicitly","Watch for numpy round-trips silently producing float64"],"tags":["dtype-validation","mamba","causal-conv1d","jit-kernel"],"backgroundTag":"unsupported-tensor-dtype","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}