sgl-project/sglang · error · Exception

VisionFlash3Attention is only available for cuda or musa

Error message

VisionFlash3Attention is only available for cuda or musa

What it means

VisionFlash3Attention relies on FlashAttention-3 cores, which only exist on CUDA GPUs and the MUSA accelerator. __init__ checks the detected platform flags _is_cuda/_is_musa and raises at construction if neither holds, so the class cannot be instantiated on ROCm/HIP, CPU, or other backends.

Source

Thrown at python/sglang/srt/layers/attention/vision.py:526

            v,
            output,
            cu_seqlens_gpu,
            seq_lens,
            max_seqlen,
            is_causal=False,
            sm_scale=softmax_scale,
        )

        return output


class VisionFlash3Attention(nn.Module):
    def __init__(
        self,
        **kwargs,
    ):
        if not (_is_cuda or _is_musa):
            raise Exception("VisionFlash3Attention is only available for cuda or musa")
        super().__init__()
        use_data_parallel = (
            kwargs["use_data_parallel"] if "use_data_parallel" in kwargs else False
        )
        self.tp_size = 1 if use_data_parallel else get_parallel().attn_tp_size

    def forward(
        self,
        q: torch.Tensor,
        k: torch.Tensor,
        v: torch.Tensor,
        cu_seqlens: torch.Tensor | SingletonCache | list | None,
        bsz: int,
        seq_len: int,
        softmax_scale: Optional[float] = None,
        forward_metadata: Optional[VisionAttentionMetadata] = None,
        **kwargs,
    ) -> torch.Tensor:

View on GitHub (pinned to 0132848349)

Solutions

Select a vision attention implementation supported by your platform (e.g. fa2/flashinfer/sdp variants available for ROCm).
Ensure PyTorch is a CUDA or MUSA build: verify torch.version.cuda / device availability before selecting flash3.
Gate backend selection on platform: choose flash3 only when torch.cuda.is_available() and not is_rocm.

Example fix

# before
attn = VisionFlash3Attention()
# after
impl = "flash3" if (torch.cuda.is_available() and not torch.version.hip) else "sdpa"
attn = VISION_ATTN_IMPLS[impl]()

Defensive patterns

Strategy: fallback

Validate before calling

import torch
flash3_ok = torch.cuda.is_available() and getattr(torch.version, "hip", None) is None
# plus MUSA check if applicable
attn_impl = "flash3" if flash3_ok else "sdpa"

Type guard

def supports_vision_flash3() -> bool:
    import torch
    return torch.cuda.is_available() and torch.version.hip is None  # or MUSA build

Try / catch

try:
    attn = VisionFlash3Attention()
except Exception:
    attn = VisionSDPAAttention()  # platform-supported fallback

Prevention

Derive the vision attention backend from detected platform, not hardcoded config.
Check torch.version.cuda in Dockerfile/CI before building flash3 images.
Test backend construction on the target hardware in smoke tests.

When it happens

Trigger: Constructing VisionFlash3Attention on a non-CUDA, non-MUSA device (e.g. ROCm build of PyTorch, CPU-only environment) — usually because the config selected attn_implementation='flash3' or fa3 for the vision encoder.

Common situations: Running a multimodal model on AMD GPUs with a config that hardcodes flash3; defaulting to fa3 in a Docker image built for the wrong platform; CI machines without GPUs importing and building the module.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/08a905dd3c435fe0. Report an issue: GitHub.