{"record":{"id":"c8b6e2a2ec76956b","repo":"sgl-project/sglang","slug":"visionflash4attention-is-only-available-for-cuda","errorCode":null,"errorMessage":"VisionFlash4Attention is only available for cuda","messagePattern":"VisionFlash4Attention is only available for cuda","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/vision.py","lineNumber":590,"sourceCode":"            max_seqlen_q=max_seqlen,\n            max_seqlen_k=max_seqlen,\n            softmax_scale=softmax_scale,\n            window_size=window_size,\n        )\n        if s_aux is not None:\n            fa_kwargs[\"sinks\"] = s_aux\n        output = flash_attn_func(q, k, v, **fa_kwargs)\n\n        return output\n\n\nclass VisionFlash4Attention(nn.Module):\n    def __init__(\n        self,\n        **kwargs,\n    ):\n        if not _is_cuda:\n            raise Exception(\"VisionFlash4Attention is only available for cuda\")\n        super().__init__()\n\n    def forward(\n        self,\n        q: torch.Tensor,\n        k: torch.Tensor,\n        v: torch.Tensor,\n        cu_seqlens: torch.Tensor | SingletonCache | None,\n        bsz: int,\n        seq_len: int,\n        softmax_scale: Optional[float] = None,\n        forward_metadata: Optional[VisionAttentionMetadata] = None,\n        **kwargs,\n    ) -> torch.Tensor:\n        r\"\"\"\n        Args:\n            cu_seqlens: [b]\n        Returns:","sourceCodeStart":572,"sourceCodeEnd":608,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/vision.py#L572-L608","documentation":"VisionFlash4Attention depends on FlashAttention-4 / cuDNN-style kernels that are CUDA-only; __init__ checks the _is_cuda platform flag and raises before super().__init__() on any non-CUDA build (including MUSA, ROCm, CPU).","triggerScenarios":"Instantiating VisionFlash4Attention on ROCm/MUSA/CPU, e.g. a config selecting flash4 (attn_implementation='flash4') for the vision encoder on a non-CUDA platform.","commonSituations":"Multimodal serving on AMD or MUSA hardware with a config inherited from a CUDA deployment; images built with ROCm PyTorch defaulting to the newest flash backend; CPU-only CI constructing the model.","solutions":["Pick a vision attention backend supported on your platform (sdpa/fa2 equivalents).","Verify torch.version.cuda is set before selecting flash4.","Make backend selection conditional on _is_cuda-style platform detection rather than hardcoding."],"exampleFix":"# before\nattn = VisionFlash4Attention()\n# after\nimpl = \"flash4\" if _is_cuda else \"sdpa\"\nattn = VISION_ATTN_IMPLS[impl]()","handlingStrategy":"fallback","validationCode":"import torch\nflash4_ok = torch.cuda.is_available() and getattr(torch.version, \"hip\", None) is None\nattn_impl = \"flash4\" if flash4_ok else \"sdpa\"","typeGuard":"def supports_vision_flash4() -> bool:\n    import torch\n    return torch.cuda.is_available() and torch.version.hip is None","tryCatchPattern":"try:\n    attn = VisionFlash4Attention()\nexcept Exception:\n    attn = VisionSDPAAttention()","preventionTips":["Make the vision backend a function of platform, model, and driver version.","Validate platform in deployment scripts before selecting flash4.","Never assume newest flash backend exists on ROCm/MUSA builds."],"tags":["sglang","vision-transformer","flash-attention-4","platform-support","hardware-compat"],"backgroundTag":"operation-not-supported-on-device","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}