sgl-project/sglang · error · RuntimeError

unsupported input for modulate_scale_shift CUDA

Error message

unsupported input for modulate_scale_shift CUDA

What it means

modulate_scale_shift_cuda is the strict fused entry point for x * (1 + scale[:, None]) + shift[:, None]; it requires can_use_modulate_scale_shift_cuda(x, scale, shift) to accept the inputs (CUDA, supported dtype, compatible shapes/strides) or it raises rather than falling back.

Source

Thrown at python/sglang/kernels/ops/diffusion/modulate/modulate_scale_shift_jit.py:99

        or scale.dim() != 2
        or shift.shape != scale.shape
        or scale.shape != (x.shape[0], x.shape[-1])
        or not (x.is_contiguous() and scale.is_contiguous() and shift.is_contiguous())
        or x.numel() == 0
    ):
        return False
    vec = _ALIGN_BYTES // x.element_size()
    return (
        x.shape[-1] % vec == 0 and _aligned(x) and _aligned(scale) and _aligned(shift)
    )


def modulate_scale_shift_cuda(
    x: torch.Tensor, scale: torch.Tensor, shift: torch.Tensor
) -> torch.Tensor:
    """Fused ``x * (1 + scale[:, None]) + shift[:, None]`` (bit-exact vs eager)."""
    if not can_use_modulate_scale_shift_cuda(x, scale, shift):
        raise RuntimeError("unsupported input for modulate_scale_shift CUDA")
    return _modulate_scale_shift_custom_op(x, scale, shift)


def modulate_scale_shift(
    x: torch.Tensor, scale: torch.Tensor, shift: torch.Tensor
) -> torch.Tensor:
    """Use the bit-exact CUDA fast path when supported, otherwise eager."""
    runtime_key = (x.device.index, x.dtype)
    if runtime_key not in _FAILED_RUNTIME_KEYS and can_use_modulate_scale_shift_cuda(
        x, scale, shift
    ):
        try:
            return modulate_scale_shift_cuda(x, scale, shift)
        except Exception as exc:
            if torch.compiler.is_compiling():
                raise
            _FAILED_RUNTIME_KEYS.add(runtime_key)
            logger.warning(

View on GitHub (pinned to 0132848349)

Solutions

1. Pre-check with can_use_modulate_scale_shift_cuda(x, scale, shift) and use the eager formula otherwise
2. Prefer the public modulate_scale_shift wrapper, which includes fallback handling
3. Ensure x is 2D+ CUDA contiguous and scale/shift are 1D with matching first dim
4. Cast dtypes to the supported set

Example fix

# before
y = modulate_scale_shift_cuda(x, s, b)
# after
if can_use_modulate_scale_shift_cuda(x, s, b):
    y = modulate_scale_shift_cuda(x, s, b)
else:
    y = x * (1 + s[:, None]) + b[:, None]

from sglang.kernels.ops.diffusion.modulate.modulate_scale_shift_jit import can_use_modulate_scale_shift_cuda
if not can_use_modulate_scale_shift_cuda(x, scale, shift):
    result = x * (1 + scale[:, None]) + shift[:, None]

def can_modulate(x, s, b) -> bool:
    return can_use_modulate_scale_shift_cuda(x, s, b)

try:
    y = modulate_scale_shift_cuda(x, scale, shift)
except RuntimeError:
    y = x * (1 + scale[:, None]) + shift[:, None]

When it happens

Trigger: Direct calls with CPU tensors, mismatched shapes (scale/shift not matching x's row count), non-contiguous inputs, or unsupported dtypes that fail the can_use check.

Common situations: Calling the CUDA entry directly from model code or tests (as in _ltx2_modulate) without pre-validating; tensors sliced from AdaLN chunk outputs with unexpected strides.

Related errors

• unsupported input for usp_merge_heads CUDA
• unsupported input for causal Conv3D cat/pad CUDA
• Unsupported modulate_scale_shift dtype: {dtype}
• unsupported input for LTX2 QKNorm split-RoPE CUDA
• Cannot find NVIDIA Math-DX (cuBLASDx) headers. Install the `

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f94b41bfcc6c3ae4. Report an issue: GitHub.