sgl-project/sglang · error · RuntimeError

ViT CUDA graph does not support attention backend: {backend}

Error message

ViT CUDA graph does not support attention backend: {backend}

What it means

The ViT CUDA graph capture only knows how to build cumulative-seqlens workspaces for the 'triton_attn' (3-entry: q, kv, max_len) and 'fa3' (2-entry: q, max_len) attention backends. Any other backend string hits this RuntimeError during create_graph.

Source

Thrown at python/sglang/srt/multimodal/vit_cuda_graph_runner.py:201

                    if layer_num in vit.fullatt_block_indexes:
                        cu_seqlens_now = cu_full
                        cu_seqlens_kk_now = cu_full_kk
                        max_len = max_full_len
                    else:
                        cu_seqlens_now = cu_window
                        cu_seqlens_kk_now = cu_window_kk
                        max_len = max_window_len
                else:
                    cu_seqlens_now = cu_full
                    cu_seqlens_kk_now = cu_full_kk
                    max_len = max_full_len

                if backend == "triton_attn":
                    cu_seq_len_ws = [cu_seqlens_now, cu_seqlens_kk_now, max_len]
                elif backend == "fa3":
                    cu_seq_len_ws = [cu_seqlens_now, max_len]
                else:
                    raise RuntimeError(
                        f"ViT CUDA graph does not support attention backend: {backend}"
                    )

                if position_embeddings is not None:
                    if layer_num == 0:
                        y = blk(
                            self.block_input[graph_key],
                            cu_seqlens=cu_seq_len_ws,
                            position_embeddings=position_embeddings,
                            output_ws=self.block_ws[graph_key],
                        )
                    else:
                        y = blk(
                            y,
                            cu_seqlens=cu_seq_len_ws,
                            position_embeddings=position_embeddings,
                            output_ws=self.block_ws[graph_key],
                        )

View on GitHub (pinned to 0132848349)

Solutions

1. Set the ViT attention backend to 'triton_attn' or 'fa3' (e.g. --mm-attention-backend triton_attn)
2. Disable ViT CUDA graph capture if the backend must be used (run without ViT graph capture)
3. Check sglang release notes/support matrix for newly supported backends and upgrade

Example fix

# before
server_args.mm_attention_backend = 'flashinfer'
# after
server_args.mm_attention_backend = 'triton_attn'

from sglang.srt.multimodal.vit_cuda_graph_runner import SUPPORTED  # conceptual
assert server_args.mm_attention_backend in ('triton_attn', 'fa3'), 'backend unsupported for ViT graph'

When it happens

Trigger: Running a multimodal ViT model with --mm-attention-backend (or environment-selected backend) set to something like 'flashinfer', 'flashmla', or a new backend, then triggering CUDA graph capture.

Common situations: Upgrading sglang where a new attention backend became default; explicitly overriding the ViT backend flag; running on hardware where fa3/triton are unavailable and a fallback is chosen.

Related errors

• deepstack_visual_indexes exists but deepstack_merger_list is
• Z-Image caption tensor must have rank 2 or 3
• {selection_error}{component_suffix}
• No compatible attention backend is available{component_suffi
• Attention backend '{selected_backend}' is not supported by t

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0f3f7067c9a48108. Report an issue: GitHub.