{"record":{"id":"cb8173716f9bc695","repo":"sgl-project/sglang","slug":"tgv-cute-ext-backend-supports-list-torch-to-cutl","errorCode":null,"errorMessage":"TGV cute_ext backend supports {list(_TORCH_TO_CUTLASS_DTYPE)}; got {dtype}.","messagePattern":"TGV cute_ext backend supports (.+?); got (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/gemm/cutedsl_bf16_gemm.py","lineNumber":1140,"sourceCode":"        dtype,\n        c_dtype,\n        cta_m,\n        cta_n,\n        cta_k,\n        num_ab_stage,\n        bool(use_2cta),\n        bool(use_pdl),\n        bool(has_bias),\n        a_leading,\n        b_leading,\n        c_leading,\n    )\n    cached = _TGV_CUTE_EXT_COMPILE_CACHE.get(key)\n    if cached is not None:\n        return cached\n\n    if dtype not in _TORCH_TO_CUTLASS_DTYPE:\n        raise ValueError(\n            f\"TGV cute_ext backend supports {list(_TORCH_TO_CUTLASS_DTYPE)}; got {dtype}.\"\n        )\n    if c_dtype not in _TORCH_TO_CUTLASS_OUT_DTYPE:\n        raise ValueError(\n            f\"TGV cute_ext output supports {list(_TORCH_TO_CUTLASS_OUT_DTYPE)}; \"\n            f\"got {c_dtype}.\"\n        )\n\n    gemm = TgvGemmCuteExtKernel(\n        acc_dtype=cutlass.Float32,\n        cta_m=cta_m,\n        cta_n=cta_n,\n        cta_k=cta_k,\n        num_ab_stage=num_ab_stage,\n        use_2cta=use_2cta,\n        use_pdl=use_pdl,\n        has_bias=has_bias,\n        out_dtype=_TORCH_TO_CUTLASS_OUT_DTYPE[c_dtype],","sourceCodeStart":1122,"sourceCodeEnd":1158,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/gemm/cutedsl_bf16_gemm.py#L1122-L1158","documentation":"The TGV cute_ext GEMM backend only accepts input dtypes present in _TORCH_TO_CUTLASS_DTYPE (bf16/fp16 mappings). Any other input dtype is rejected before kernel compilation.","triggerScenarios":"Calling _run_tgv with x/weight in fp8, fp32, or int dtypes.","commonSituations":"Routing a quantized (fp8) linear layer to the TGV bf16 path, or accidentally upcasting inputs to fp32.","solutions":["Ensure x and weight are bfloat16 (or the other supported dtype in _TORCH_TO_CUTLASS_DTYPE).","Route fp8 GEMMs to the fp8-specific kernels instead.","Check upstream dtype before dispatch and cast if the math allows."],"exampleFix":"// before\nout = _run_tgv(x.float(), w.float(), None)\n// after\nout = _run_tgv(x.bfloat16(), w.bfloat16(), None)","handlingStrategy":"type-guard","validationCode":"assert x.dtype in _TORCH_TO_CUTLASS_DTYPE and weight.dtype in _TORCH_TO_CUTLASS_DTYPE","typeGuard":"def tgv_input_ok(t: torch.Tensor) -> bool:\n    return t.dtype in _TORCH_TO_CUTLASS_DTYPE","tryCatchPattern":null,"preventionTips":["Route fp8 layers to fp8 kernels, not TGV bf16.","Assert dtypes at dispatch time."],"tags":["gemm","tgv","dtype","cutlass"],"backgroundTag":"unsupported-dtype-for-kernel","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}