sgl-project/sglang · error · ValueError

TGV cute_ext output supports

Error message

TGV cute_ext output supports {list(_TORCH_TO_CUTLASS_OUT_DTYPE)}; got {c_dtype}.

What it means

Output dtype support check for the TGV cute_ext GEMM: the requested output dtype is not among the mapped output types in _TORCH_TO_CUTLASS_OUT_DTYPE. Use a supported output dtype (e.g. the same as the input) or fall back to another backend.

Solutions

  1. Use bfloat16 or float32 output.
  2. Add an explicit cast of the result after the GEMM if fp16 is needed downstream.
  3. Check _TORCH_TO_CUTLASS_OUT_DTYPE keys before configuring.

Example fix

// before
out = torch.empty(M, N, dtype=torch.float16, device='cuda')
// after
out = torch.empty(M, N, dtype=torch.bfloat16, device='cuda')
# cast later if needed: out.half()
Defensive patterns

Strategy: type-guard

Validate before calling

assert out.dtype in _TORCH_TO_CUTLASS_OUT_DTYPE

Type guard

def tgv_output_ok(t: torch.Tensor) -> bool:
    return t.dtype in _TORCH_TO_CUTLASS_OUT_DTYPE

Prevention

When it happens

Trigger: Calling _run_tgv with an out tensor or c_dtype of torch.float16, torch.float8, or int types.

Common situations: Configs tuned for other backends where fp16 output is allowed, then reused with the TGV path.

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8062d21e455190d8. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/kernels/ops/gemm/cutedsl_bf16_gemm.py:1144

        cta_k,
        num_ab_stage,
        bool(use_2cta),
        bool(use_pdl),
        bool(has_bias),
        a_leading,
        b_leading,
        c_leading,
    )
    cached = _TGV_CUTE_EXT_COMPILE_CACHE.get(key)
    if cached is not None:
        return cached

    if dtype not in _TORCH_TO_CUTLASS_DTYPE:
        raise ValueError(
            f"TGV cute_ext backend supports {list(_TORCH_TO_CUTLASS_DTYPE)}; got {dtype}."
        )
    if c_dtype not in _TORCH_TO_CUTLASS_OUT_DTYPE:
        raise ValueError(
            f"TGV cute_ext output supports {list(_TORCH_TO_CUTLASS_OUT_DTYPE)}; "
            f"got {c_dtype}."
        )

    gemm = TgvGemmCuteExtKernel(
        acc_dtype=cutlass.Float32,
        cta_m=cta_m,
        cta_n=cta_n,
        cta_k=cta_k,
        num_ab_stage=num_ab_stage,
        use_2cta=use_2cta,
        use_pdl=use_pdl,
        has_bias=has_bias,
        out_dtype=_TORCH_TO_CUTLASS_OUT_DTYPE[c_dtype],
    )

    a_, b_, c_, bias_ = _make_compile_repr_tensors(
        dtype,

View on GitHub (pinned to 0132848349)