{"record":{"id":"f94b41bfcc6c3ae4","repo":"sgl-project/sglang","slug":"unsupported-input-for-modulate-scale-shift-cuda","errorCode":null,"errorMessage":"unsupported input for modulate_scale_shift CUDA","messagePattern":"unsupported input for modulate_scale_shift CUDA","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/diffusion/modulate/modulate_scale_shift_jit.py","lineNumber":99,"sourceCode":"        or scale.dim() != 2\n        or shift.shape != scale.shape\n        or scale.shape != (x.shape[0], x.shape[-1])\n        or not (x.is_contiguous() and scale.is_contiguous() and shift.is_contiguous())\n        or x.numel() == 0\n    ):\n        return False\n    vec = _ALIGN_BYTES // x.element_size()\n    return (\n        x.shape[-1] % vec == 0 and _aligned(x) and _aligned(scale) and _aligned(shift)\n    )\n\n\ndef modulate_scale_shift_cuda(\n    x: torch.Tensor, scale: torch.Tensor, shift: torch.Tensor\n) -> torch.Tensor:\n    \"\"\"Fused ``x * (1 + scale[:, None]) + shift[:, None]`` (bit-exact vs eager).\"\"\"\n    if not can_use_modulate_scale_shift_cuda(x, scale, shift):\n        raise RuntimeError(\"unsupported input for modulate_scale_shift CUDA\")\n    return _modulate_scale_shift_custom_op(x, scale, shift)\n\n\ndef modulate_scale_shift(\n    x: torch.Tensor, scale: torch.Tensor, shift: torch.Tensor\n) -> torch.Tensor:\n    \"\"\"Use the bit-exact CUDA fast path when supported, otherwise eager.\"\"\"\n    runtime_key = (x.device.index, x.dtype)\n    if runtime_key not in _FAILED_RUNTIME_KEYS and can_use_modulate_scale_shift_cuda(\n        x, scale, shift\n    ):\n        try:\n            return modulate_scale_shift_cuda(x, scale, shift)\n        except Exception as exc:\n            if torch.compiler.is_compiling():\n                raise\n            _FAILED_RUNTIME_KEYS.add(runtime_key)\n            logger.warning(","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/diffusion/modulate/modulate_scale_shift_jit.py#L81-L117","documentation":"modulate_scale_shift_cuda is the strict fused entry point for x * (1 + scale[:, None]) + shift[:, None]; it requires can_use_modulate_scale_shift_cuda(x, scale, shift) to accept the inputs (CUDA, supported dtype, compatible shapes/strides) or it raises rather than falling back.","triggerScenarios":"Direct calls with CPU tensors, mismatched shapes (scale/shift not matching x's row count), non-contiguous inputs, or unsupported dtypes that fail the can_use check.","commonSituations":"Calling the CUDA entry directly from model code or tests (as in _ltx2_modulate) without pre-validating; tensors sliced from AdaLN chunk outputs with unexpected strides.","solutions":["Pre-check with can_use_modulate_scale_shift_cuda(x, scale, shift) and use the eager formula otherwise","Prefer the public modulate_scale_shift wrapper, which includes fallback handling","Ensure x is 2D+ CUDA contiguous and scale/shift are 1D with matching first dim","Cast dtypes to the supported set"],"exampleFix":"# before\ny = modulate_scale_shift_cuda(x, s, b)\n# after\nif can_use_modulate_scale_shift_cuda(x, s, b):\n    y = modulate_scale_shift_cuda(x, s, b)\nelse:\n    y = x * (1 + s[:, None]) + b[:, None]","handlingStrategy":"fallback","validationCode":"from sglang.kernels.ops.diffusion.modulate.modulate_scale_shift_jit import can_use_modulate_scale_shift_cuda\nif not can_use_modulate_scale_shift_cuda(x, scale, shift):\n    result = x * (1 + scale[:, None]) + shift[:, None]","typeGuard":"def can_modulate(x, s, b) -> bool:\n    return can_use_modulate_scale_shift_cuda(x, s, b)","tryCatchPattern":"try:\n    y = modulate_scale_shift_cuda(x, scale, shift)\nexcept RuntimeError:\n    y = x * (1 + scale[:, None]) + shift[:, None]","preventionTips":["Use the public modulate_scale_shift wrapper","Validate shape/dtype/device before the fused entry"],"tags":["cuda","modulate","fallback","input-validation"],"backgroundTag":"tensor-input-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}