{"record":{"id":"0f3f7067c9a48108","repo":"sgl-project/sglang","slug":"vit-cuda-graph-does-not-support-attention-backend","errorCode":null,"errorMessage":"ViT CUDA graph does not support attention backend: {backend}","messagePattern":"ViT CUDA graph does not support attention backend: (.+?)","errorType":"error_code","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/multimodal/vit_cuda_graph_runner.py","lineNumber":201,"sourceCode":"                    if layer_num in vit.fullatt_block_indexes:\n                        cu_seqlens_now = cu_full\n                        cu_seqlens_kk_now = cu_full_kk\n                        max_len = max_full_len\n                    else:\n                        cu_seqlens_now = cu_window\n                        cu_seqlens_kk_now = cu_window_kk\n                        max_len = max_window_len\n                else:\n                    cu_seqlens_now = cu_full\n                    cu_seqlens_kk_now = cu_full_kk\n                    max_len = max_full_len\n\n                if backend == \"triton_attn\":\n                    cu_seq_len_ws = [cu_seqlens_now, cu_seqlens_kk_now, max_len]\n                elif backend == \"fa3\":\n                    cu_seq_len_ws = [cu_seqlens_now, max_len]\n                else:\n                    raise RuntimeError(\n                        f\"ViT CUDA graph does not support attention backend: {backend}\"\n                    )\n\n                if position_embeddings is not None:\n                    if layer_num == 0:\n                        y = blk(\n                            self.block_input[graph_key],\n                            cu_seqlens=cu_seq_len_ws,\n                            position_embeddings=position_embeddings,\n                            output_ws=self.block_ws[graph_key],\n                        )\n                    else:\n                        y = blk(\n                            y,\n                            cu_seqlens=cu_seq_len_ws,\n                            position_embeddings=position_embeddings,\n                            output_ws=self.block_ws[graph_key],\n                        )","sourceCodeStart":183,"sourceCodeEnd":219,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/multimodal/vit_cuda_graph_runner.py#L183-L219","documentation":"The ViT CUDA graph capture only knows how to build cumulative-seqlens workspaces for the 'triton_attn' (3-entry: q, kv, max_len) and 'fa3' (2-entry: q, max_len) attention backends. Any other backend string hits this RuntimeError during create_graph.","triggerScenarios":"Running a multimodal ViT model with --mm-attention-backend (or environment-selected backend) set to something like 'flashinfer', 'flashmla', or a new backend, then triggering CUDA graph capture.","commonSituations":"Upgrading sglang where a new attention backend became default; explicitly overriding the ViT backend flag; running on hardware where fa3/triton are unavailable and a fallback is chosen.","solutions":["Set the ViT attention backend to 'triton_attn' or 'fa3' (e.g. --mm-attention-backend triton_attn)","Disable ViT CUDA graph capture if the backend must be used (run without ViT graph capture)","Check sglang release notes/support matrix for newly supported backends and upgrade"],"exampleFix":"# before\nserver_args.mm_attention_backend = 'flashinfer'\n# after\nserver_args.mm_attention_backend = 'triton_attn'","handlingStrategy":"validation","validationCode":"from sglang.srt.multimodal.vit_cuda_graph_runner import SUPPORTED  # conceptual\nassert server_args.mm_attention_backend in ('triton_attn', 'fa3'), 'backend unsupported for ViT graph'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pin a known-supported ViT attention backend in deployment configs","Re-check supported backends when upgrading sglang or changing GPU platforms"],"tags":["multimodal","vit","cuda-graph","attention-backend"],"backgroundTag":"unsupported-backend","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}