sgl-project/sglang · error · Exception

VisionFlash4Attention is only available for cuda

Error message

VisionFlash4Attention is only available for cuda

What it means

VisionFlash4Attention depends on FlashAttention-4 / cuDNN-style kernels that are CUDA-only; __init__ checks the _is_cuda platform flag and raises before super().__init__() on any non-CUDA build (including MUSA, ROCm, CPU).

Source

Thrown at python/sglang/srt/layers/attention/vision.py:590

            max_seqlen_q=max_seqlen,
            max_seqlen_k=max_seqlen,
            softmax_scale=softmax_scale,
            window_size=window_size,
        )
        if s_aux is not None:
            fa_kwargs["sinks"] = s_aux
        output = flash_attn_func(q, k, v, **fa_kwargs)

        return output


class VisionFlash4Attention(nn.Module):
    def __init__(
        self,
        **kwargs,
    ):
        if not _is_cuda:
            raise Exception("VisionFlash4Attention is only available for cuda")
        super().__init__()

    def forward(
        self,
        q: torch.Tensor,
        k: torch.Tensor,
        v: torch.Tensor,
        cu_seqlens: torch.Tensor | SingletonCache | None,
        bsz: int,
        seq_len: int,
        softmax_scale: Optional[float] = None,
        forward_metadata: Optional[VisionAttentionMetadata] = None,
        **kwargs,
    ) -> torch.Tensor:
        r"""
        Args:
            cu_seqlens: [b]
        Returns:

View on GitHub (pinned to 0132848349)

Solutions

1. Pick a vision attention backend supported on your platform (sdpa/fa2 equivalents).
2. Verify torch.version.cuda is set before selecting flash4.
3. Make backend selection conditional on _is_cuda-style platform detection rather than hardcoding.

Example fix

# before
attn = VisionFlash4Attention()
# after
impl = "flash4" if _is_cuda else "sdpa"
attn = VISION_ATTN_IMPLS[impl]()

import torch
flash4_ok = torch.cuda.is_available() and getattr(torch.version, "hip", None) is None
attn_impl = "flash4" if flash4_ok else "sdpa"

def supports_vision_flash4() -> bool:
    import torch
    return torch.cuda.is_available() and torch.version.hip is None

try:
    attn = VisionFlash4Attention()
except Exception:
    attn = VisionSDPAAttention()

When it happens

Trigger: Instantiating VisionFlash4Attention on ROCm/MUSA/CPU, e.g. a config selecting flash4 (attn_implementation='flash4') for the vision encoder on a non-CUDA platform.

Common situations: Multimodal serving on AMD or MUSA hardware with a config inherited from a CUDA deployment; images built with ROCm PyTorch defaulting to the newest flash backend; CPU-only CI constructing the model.

Related errors

• VisionFlash3Attention is only available for cuda or musa
• VisionFlashInferAttention is only available for cuda
• Decode context parallel (decode_context_parallel_size > 1) i
• MXFP8 KV cache requires the FA4 backend.
• MXFP8 KV cache requires per-token Q scales (q_descale) from

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c8b6e2a2ec76956b. Report an issue: GitHub.