sgl-project/sglang · error · ValueError

Unsupported fused vision RoPE inputs: q={q.shape}/{q.dtype}/

Error message

Unsupported fused vision RoPE inputs: q={q.shape}/{q.dtype}/{q.device}, k={k.shape}/{k.dtype}/{k.device}, freqs={freqs_cis.shape}/{freqs_cis.dtype}/{freqs_cis.device}

What it means

apply_fused_qk_complex_rope only supports a specific contract checked by can_use_fused_qk_complex_rope: q and k must be same-shape CUDA bf16/fp16 tensors of ndim>=3 with even last dim, freqs_cis must be complex64 with shape q.shape[:-2] + (q.shape[-1]//2,) on the same device, and the GPU must be compute capability >= 9 (Hopper/Blackwell). The message dumps all shapes/dtypes/devices so mismatches are visible.

Source

Thrown at python/sglang/kernels/ops/attention/vision_rope.py:116

    major, _ = torch.cuda.get_device_capability(q.device)
    return major >= 9


def apply_fused_qk_complex_rope(
    q: torch.Tensor,
    k: torch.Tensor,
    freqs_cis: torch.Tensor,
) -> Tuple[torch.Tensor, torch.Tensor]:
    """Rotate interleaved Q/K pairs with one kernel.

    ``q`` and ``k`` may be strided views of an interleaved QKV projection. The
    output is contiguous, matching the native complex-multiply implementation.
    The token count remains a runtime kernel argument so random image sizes do
    not create new Triton specializations.
    """

    if not can_use_fused_qk_complex_rope(q, k, freqs_cis):
        raise ValueError(
            "Unsupported fused vision RoPE inputs: "
            f"q={q.shape}/{q.dtype}/{q.device}, "
            f"k={k.shape}/{k.dtype}/{k.device}, "
            f"freqs={freqs_cis.shape}/{freqs_cis.dtype}/{freqs_cis.device}"
        )

    original_shape = q.shape
    # Preserve the interleaved QKV token stride when the token dimension is 1.
    # ``view(-1, ...)`` is otherwise free to collapse that singleton stride,
    # producing a different Triton specialization from real image requests.
    q_flat = q if q.ndim == 3 else q.view(-1, q.shape[-2], q.shape[-1])
    k_flat = k if k.ndim == 3 else k.view(-1, k.shape[-2], k.shape[-1])
    freqs = torch.view_as_real(freqs_cis).view(-1, q.shape[-1] // 2, 2)
    q_out = torch.empty(q_flat.shape, dtype=q.dtype, device=q.device)
    k_out = torch.empty(k_flat.shape, dtype=k.dtype, device=k.device)

    block = 128
    n_pairs = q_flat.numel() // 2

View on GitHub (pinned to 0132848349)

Solutions

Guard with if can_use_fused_qk_complex_rope(q, k, freqs_cis) and fall back to the portable complex-multiply path otherwise
Ensure freqs_cis = freqs_cis.to(torch.complex64) and its shape equals q.shape[:-2] + (q.shape[-1]//2,)
On GPUs below SM90, use the non-fused reference implementation

Example fix

// before
q, k = apply_fused_qk_complex_rope(q, k, freqs_cis)
// after
if can_use_fused_qk_complex_rope(q, k, freqs_cis):
    q, k = apply_fused_qk_complex_rope(q, k, freqs_cis)
else:
    q, k = portable_complex_rope(q, k, freqs_cis)

Defensive patterns

Strategy: fallback

Validate before calling

from sglang.kernels.ops.attention.vision_rope import can_use_fused_qk_complex_rope
if not can_use_fused_qk_complex_rope(q, k, freqs_cis):
    q, k = portable_complex_rope(q, k, freqs_cis)  # eager fallback

Type guard

def supports_fused_rope(q, k, f) -> bool:
    return (
        q.is_cuda and k.is_cuda and f.is_cuda
        and q.device == k.device == f.device
        and q.dtype == k.dtype and q.dtype in (torch.bfloat16, torch.float16)
        and f.dtype == torch.complex64 and q.shape == k.shape and q.ndim >= 3
        and q.shape[-1] % 2 == 0
        and f.shape == q.shape[:-2] + (q.shape[-1] // 2,)
        and torch.cuda.get_device_capability(q.device)[0] >= 9
    )

Try / catch

try:
    q, k = apply_fused_qk_complex_rope(q, k, freqs_cis)
except ValueError:
    q, k = portable_complex_rope(q, k, freqs_cis)

Prevention

Keep the can_use predicate next to every fused-path call site
Check torch.cuda.get_device_capability before enabling fused vision RoPE

When it happens

Trigger: Calling apply_fused_qk_complex_rope on pre-Ampere/Hopper GPUs (major < 9), with fp32 q/k, mismatched q/k shapes, real-valued (non-complex) freqs, or freqs_cis whose token count or head_dim/2 length doesn't line up with q.

Common situations: Running a vision model (e.g. Kimi MoonViT) on an A100 or T4 where the fused path requires SM90+, or passing freqs computed for a different resolution/head_dim than q/k.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/66654729cfd3113b. Report an issue: GitHub.