sgl-project/sglang · error · ValueError

TGV cute_ext output supports {list(_TORCH_TO_CUTLASS_OUT_DTY

Error message

TGV cute_ext output supports {list(_TORCH_TO_CUTLASS_OUT_DTYPE)}; got {c_dtype}.

What it means

The TGV cute_ext backend constrains the output/accumulate dtype to _TORCH_TO_CUTLASS_OUT_DTYPE (bf16/fp32). Requesting e.g. fp16 or fp8 output raises before compilation.

Source

Thrown at python/sglang/kernels/ops/gemm/cutedsl_bf16_gemm.py:1144

        cta_k,
        num_ab_stage,
        bool(use_2cta),
        bool(use_pdl),
        bool(has_bias),
        a_leading,
        b_leading,
        c_leading,
    )
    cached = _TGV_CUTE_EXT_COMPILE_CACHE.get(key)
    if cached is not None:
        return cached

    if dtype not in _TORCH_TO_CUTLASS_DTYPE:
        raise ValueError(
            f"TGV cute_ext backend supports {list(_TORCH_TO_CUTLASS_DTYPE)}; got {dtype}."
        )
    if c_dtype not in _TORCH_TO_CUTLASS_OUT_DTYPE:
        raise ValueError(
            f"TGV cute_ext output supports {list(_TORCH_TO_CUTLASS_OUT_DTYPE)}; "
            f"got {c_dtype}."
        )

    gemm = TgvGemmCuteExtKernel(
        acc_dtype=cutlass.Float32,
        cta_m=cta_m,
        cta_n=cta_n,
        cta_k=cta_k,
        num_ab_stage=num_ab_stage,
        use_2cta=use_2cta,
        use_pdl=use_pdl,
        has_bias=has_bias,
        out_dtype=_TORCH_TO_CUTLASS_OUT_DTYPE[c_dtype],
    )

    a_, b_, c_, bias_ = _make_compile_repr_tensors(
        dtype,

View on GitHub (pinned to 0132848349)

Solutions

1. Use bfloat16 or float32 output.
2. Add an explicit cast of the result after the GEMM if fp16 is needed downstream.
3. Check _TORCH_TO_CUTLASS_OUT_DTYPE keys before configuring.

Example fix

// before
out = torch.empty(M, N, dtype=torch.float16, device='cuda')
// after
out = torch.empty(M, N, dtype=torch.bfloat16, device='cuda')
# cast later if needed: out.half()

assert out.dtype in _TORCH_TO_CUTLASS_OUT_DTYPE

def tgv_output_ok(t: torch.Tensor) -> bool:
    return t.dtype in _TORCH_TO_CUTLASS_OUT_DTYPE

When it happens

Trigger: Calling _run_tgv with an out tensor or c_dtype of torch.float16, torch.float8, or int types.

Common situations: Configs tuned for other backends where fp16 output is allowed, then reused with the TGV path.

Related errors

• TGV cute_ext backend supports {list(_TORCH_TO_CUTLASS_DTYPE)
• TGV cute_ext tactic {tactic} out of range [0, {len(_TGV_CUTE
• Cannot find CUTLASS headers required for JIT compilation. Pl
• Unsupported tcgen05 MMA op kind: {type(op).__name__}
• SplitKV partial output (mO) must be Float32

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8062d21e455190d8. Report an issue: GitHub.