sgl-project/sglang · error · RuntimeError

unsupported dtype for causal Conv3D cat/pad: {x.dtype}

Error message

unsupported dtype for causal Conv3D cat/pad: {x.dtype}

What it means

The fused causal Conv3D cat/pad CUDA kernel only supports a fixed set of dtypes (defined in _SUPPORTED_DTYPES in causal_conv3d_cat_pad_jit.py). When the input tensor x has a dtype outside that set, the wrapper raises immediately rather than compiling a kernel variant for it.

Source

Thrown at python/sglang/kernels/ops/diffusion/layout/causal_conv3d_cat_pad_jit.py:104

        x,
        cache_x,
        pad_w_left,
        pad_w_right,
        pad_h_top,
        pad_h_bottom,
        pad_d_left,
        pad_d_right,
    )
    return out


def fused_causal_conv3d_cat_pad_cuda(
    x: torch.Tensor,
    cache_x: torch.Tensor,
    padding: list[int] | tuple[int, ...],
) -> torch.Tensor:
    if x.dtype not in _SUPPORTED_DTYPES:
        raise RuntimeError(f"unsupported dtype for causal Conv3D cat/pad: {x.dtype}")
    if not torch.compiler.is_compiling():
        if (
            not x.is_cuda
            or not cache_x.is_cuda
            or x.dim() != 5
            or cache_x.dim() != 5
            or not x.is_contiguous()
            or not cache_x.is_contiguous()
            or not can_use_fused_causal_conv3d_cat_pad_cuda(x, cache_x, padding)
        ):
            raise RuntimeError("unsupported input for causal Conv3D cat/pad CUDA")
    return _causal_conv3d_cat_pad_custom_op(x, cache_x, *padding)


def can_use_fused_causal_conv3d_cat_pad_cuda(
    x: torch.Tensor,
    cache_x: torch.Tensor,
    padding: list[int] | tuple[int, ...],

View on GitHub (pinned to 0132848349)

Solutions

1. Check x.dtype against _SUPPORTED_DTYPES (fp16/bf16/fp32) before calling
2. Cast the input: x = x.to(torch.bfloat16) (and cache_x likewise)
3. Fall back to the eager PyTorch path (torch.cat + F.pad) for unsupported dtypes
4. Extend _SUPPORTED_DTYPES and regenerate the JIT kernel if you truly need a new dtype

Example fix

# before
out = fused_causal_conv3d_cat_pad_cuda(x_fp64, cache, padding)
# after
x = x.to(torch.bfloat16); cache = cache.to(torch.bfloat16)
out = fused_causal_conv3d_cat_pad_cuda(x, cache, padding)

from sglang.kernels.ops.diffusion.layout.causal_conv3d_cat_pad_jit import _SUPPORTED_DTYPES
if x.dtype not in _SUPPORTED_DTYPES:
    x = x.to(torch.bfloat16); cache_x = cache_x.to(torch.bfloat16)

def conv3d_dtype_ok(x: torch.Tensor) -> bool:
    return x.dtype in (torch.float16, torch.bfloat16, torch.float32)

When it happens

Trigger: Calling fused_causal_conv3d_cat_pad_cuda (directly or via fused_causal_conv3d_cat_pad) with x in an unsupported dtype such as float64 or an integer/bool dtype, while cache_x is concatenated onto x for a causal Conv3D layer.

Common situations: Running a diffusion model (e.g. LTX2-style video models) whose activations were cast to fp64 for debugging, or feeding fp8/int activations; mixing new dtypes not yet added to the JIT kernel's supported list.

Related errors

• SplitKV partial output (mO) must be Float32
• All tensors must have the same data type
• Only Float16 or BFloat16 is supported
• LSE tensor must be Float32
• cu_seqlens_q tensor must be Int32

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a965e0c0cfdc473a. Report an issue: GitHub.