{"record":{"id":"08a905dd3c435fe0","repo":"sgl-project/sglang","slug":"visionflash3attention-is-only-available-for-cuda-o","errorCode":null,"errorMessage":"VisionFlash3Attention is only available for cuda or musa","messagePattern":"VisionFlash3Attention is only available for cuda or musa","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/vision.py","lineNumber":526,"sourceCode":"            v,\n            output,\n            cu_seqlens_gpu,\n            seq_lens,\n            max_seqlen,\n            is_causal=False,\n            sm_scale=softmax_scale,\n        )\n\n        return output\n\n\nclass VisionFlash3Attention(nn.Module):\n    def __init__(\n        self,\n        **kwargs,\n    ):\n        if not (_is_cuda or _is_musa):\n            raise Exception(\"VisionFlash3Attention is only available for cuda or musa\")\n        super().__init__()\n        use_data_parallel = (\n            kwargs[\"use_data_parallel\"] if \"use_data_parallel\" in kwargs else False\n        )\n        self.tp_size = 1 if use_data_parallel else get_parallel().attn_tp_size\n\n    def forward(\n        self,\n        q: torch.Tensor,\n        k: torch.Tensor,\n        v: torch.Tensor,\n        cu_seqlens: torch.Tensor | SingletonCache | list | None,\n        bsz: int,\n        seq_len: int,\n        softmax_scale: Optional[float] = None,\n        forward_metadata: Optional[VisionAttentionMetadata] = None,\n        **kwargs,\n    ) -> torch.Tensor:","sourceCodeStart":508,"sourceCodeEnd":544,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/vision.py#L508-L544","documentation":"VisionFlash3Attention relies on FlashAttention-3 cores, which only exist on CUDA GPUs and the MUSA accelerator. __init__ checks the detected platform flags _is_cuda/_is_musa and raises at construction if neither holds, so the class cannot be instantiated on ROCm/HIP, CPU, or other backends.","triggerScenarios":"Constructing VisionFlash3Attention on a non-CUDA, non-MUSA device (e.g. ROCm build of PyTorch, CPU-only environment) — usually because the config selected attn_implementation='flash3' or fa3 for the vision encoder.","commonSituations":"Running a multimodal model on AMD GPUs with a config that hardcodes flash3; defaulting to fa3 in a Docker image built for the wrong platform; CI machines without GPUs importing and building the module.","solutions":["Select a vision attention implementation supported by your platform (e.g. fa2/flashinfer/sdp variants available for ROCm).","Ensure PyTorch is a CUDA or MUSA build: verify torch.version.cuda / device availability before selecting flash3.","Gate backend selection on platform: choose flash3 only when torch.cuda.is_available() and not is_rocm."],"exampleFix":"# before\nattn = VisionFlash3Attention()\n# after\nimpl = \"flash3\" if (torch.cuda.is_available() and not torch.version.hip) else \"sdpa\"\nattn = VISION_ATTN_IMPLS[impl]()","handlingStrategy":"fallback","validationCode":"import torch\nflash3_ok = torch.cuda.is_available() and getattr(torch.version, \"hip\", None) is None\n# plus MUSA check if applicable\nattn_impl = \"flash3\" if flash3_ok else \"sdpa\"","typeGuard":"def supports_vision_flash3() -> bool:\n    import torch\n    return torch.cuda.is_available() and torch.version.hip is None  # or MUSA build","tryCatchPattern":"try:\n    attn = VisionFlash3Attention()\nexcept Exception:\n    attn = VisionSDPAAttention()  # platform-supported fallback","preventionTips":["Derive the vision attention backend from detected platform, not hardcoded config.","Check torch.version.cuda in Dockerfile/CI before building flash3 images.","Test backend construction on the target hardware in smoke tests."],"tags":["sglang","vision-transformer","flash-attention","platform-support","hardware-compat"],"backgroundTag":"operation-not-supported-on-device","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}