{"record":{"id":"66654729cfd3113b","repo":"sgl-project/sglang","slug":"unsupported-fused-vision-rope-inputs-q-q-shape","errorCode":null,"errorMessage":"Unsupported fused vision RoPE inputs: q={q.shape}/{q.dtype}/{q.device}, k={k.shape}/{k.dtype}/{k.device}, freqs={freqs_cis.shape}/{freqs_cis.dtype}/{freqs_cis.device}","messagePattern":"Unsupported fused vision RoPE inputs: q=(.+?)/(.+?)/(.+?), k=(.+?)/(.+?)/(.+?), freqs=(.+?)/(.+?)/(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/vision_rope.py","lineNumber":116,"sourceCode":"    major, _ = torch.cuda.get_device_capability(q.device)\n    return major >= 9\n\n\ndef apply_fused_qk_complex_rope(\n    q: torch.Tensor,\n    k: torch.Tensor,\n    freqs_cis: torch.Tensor,\n) -> Tuple[torch.Tensor, torch.Tensor]:\n    \"\"\"Rotate interleaved Q/K pairs with one kernel.\n\n    ``q`` and ``k`` may be strided views of an interleaved QKV projection. The\n    output is contiguous, matching the native complex-multiply implementation.\n    The token count remains a runtime kernel argument so random image sizes do\n    not create new Triton specializations.\n    \"\"\"\n\n    if not can_use_fused_qk_complex_rope(q, k, freqs_cis):\n        raise ValueError(\n            \"Unsupported fused vision RoPE inputs: \"\n            f\"q={q.shape}/{q.dtype}/{q.device}, \"\n            f\"k={k.shape}/{k.dtype}/{k.device}, \"\n            f\"freqs={freqs_cis.shape}/{freqs_cis.dtype}/{freqs_cis.device}\"\n        )\n\n    original_shape = q.shape\n    # Preserve the interleaved QKV token stride when the token dimension is 1.\n    # ``view(-1, ...)`` is otherwise free to collapse that singleton stride,\n    # producing a different Triton specialization from real image requests.\n    q_flat = q if q.ndim == 3 else q.view(-1, q.shape[-2], q.shape[-1])\n    k_flat = k if k.ndim == 3 else k.view(-1, k.shape[-2], k.shape[-1])\n    freqs = torch.view_as_real(freqs_cis).view(-1, q.shape[-1] // 2, 2)\n    q_out = torch.empty(q_flat.shape, dtype=q.dtype, device=q.device)\n    k_out = torch.empty(k_flat.shape, dtype=k.dtype, device=k.device)\n\n    block = 128\n    n_pairs = q_flat.numel() // 2","sourceCodeStart":98,"sourceCodeEnd":134,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/vision_rope.py#L98-L134","documentation":"apply_fused_qk_complex_rope only supports a specific contract checked by can_use_fused_qk_complex_rope: q and k must be same-shape CUDA bf16/fp16 tensors of ndim>=3 with even last dim, freqs_cis must be complex64 with shape q.shape[:-2] + (q.shape[-1]//2,) on the same device, and the GPU must be compute capability >= 9 (Hopper/Blackwell). The message dumps all shapes/dtypes/devices so mismatches are visible.","triggerScenarios":"Calling apply_fused_qk_complex_rope on pre-Ampere/Hopper GPUs (major < 9), with fp32 q/k, mismatched q/k shapes, real-valued (non-complex) freqs, or freqs_cis whose token count or head_dim/2 length doesn't line up with q.","commonSituations":"Running a vision model (e.g. Kimi MoonViT) on an A100 or T4 where the fused path requires SM90+, or passing freqs computed for a different resolution/head_dim than q/k.","solutions":["Guard with if can_use_fused_qk_complex_rope(q, k, freqs_cis) and fall back to the portable complex-multiply path otherwise","Ensure freqs_cis = freqs_cis.to(torch.complex64) and its shape equals q.shape[:-2] + (q.shape[-1]//2,)","On GPUs below SM90, use the non-fused reference implementation"],"exampleFix":"// before\nq, k = apply_fused_qk_complex_rope(q, k, freqs_cis)\n// after\nif can_use_fused_qk_complex_rope(q, k, freqs_cis):\n    q, k = apply_fused_qk_complex_rope(q, k, freqs_cis)\nelse:\n    q, k = portable_complex_rope(q, k, freqs_cis)","handlingStrategy":"fallback","validationCode":"from sglang.kernels.ops.attention.vision_rope import can_use_fused_qk_complex_rope\nif not can_use_fused_qk_complex_rope(q, k, freqs_cis):\n    q, k = portable_complex_rope(q, k, freqs_cis)  # eager fallback","typeGuard":"def supports_fused_rope(q, k, f) -> bool:\n    return (\n        q.is_cuda and k.is_cuda and f.is_cuda\n        and q.device == k.device == f.device\n        and q.dtype == k.dtype and q.dtype in (torch.bfloat16, torch.float16)\n        and f.dtype == torch.complex64 and q.shape == k.shape and q.ndim >= 3\n        and q.shape[-1] % 2 == 0\n        and f.shape == q.shape[:-2] + (q.shape[-1] // 2,)\n        and torch.cuda.get_device_capability(q.device)[0] >= 9\n    )","tryCatchPattern":"try:\n    q, k = apply_fused_qk_complex_rope(q, k, freqs_cis)\nexcept ValueError:\n    q, k = portable_complex_rope(q, k, freqs_cis)","preventionTips":["Keep the can_use predicate next to every fused-path call site","Check torch.cuda.get_device_capability before enabling fused vision RoPE"],"tags":["vision-rope","triton","gpu-capability","tensor-validation"],"backgroundTag":"unsupported-kernel-input-contract","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}