sgl-project/sglang · error · ValueError

f"flash attention version {fa_ver} is not supported."

Error message

f"flash attention version {fa_ver} is not supported."

What it means

FlashAttention.forward dispatches among flash-attention kernel versions (fa_ver); if the requested version is not one of the implemented dispatch branches, it raises ValueError naming the unsupported version.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/flash_attn.py:449

                    ver=fa_ver,
                )
                return out_tensor, softmax_lse
            out_tensor = flash_attn_varlen_func_op(
                q=query,
                k=key,
                v=value,
                cu_seqlens_q=None,
                cu_seqlens_k=None,
                max_seqlen_q=max_seqlen_q,
                max_seqlen_k=max_seqlen_k,
                softmax_scale=self.softmax_scale,
                causal=self.causal,
                return_softmax_lse=False,
                ver=fa_ver,
            )
            return out_tensor

        raise ValueError(f"flash attention version {fa_ver} is not supported.")

    def forward_varlen(
        self,
        query: torch.Tensor,
        key: torch.Tensor,
        value: torch.Tensor,
        *,
        cu_seqlens: torch.Tensor,
        max_seqlen: int,
        cu_seqlens_host: tuple[int, ...] | None = None,
    ) -> torch.Tensor:
        del cu_seqlens_host
        output = flash_attn_varlen_func(
            query,
            key,
            value,
            cu_seqlens_q=cu_seqlens,
            cu_seqlens_k=cu_seqlens,

View on GitHub (pinned to 0132848349)

Solutions

1. Check the dispatch branches in flash_attn.py forward() and remove the override so a supported ver is used
2. Downgrade flash-attn to a major version supported by this SGLang release, or upgrade SGLang
3. If you genuinely need the new version, add a dispatch case and verify numerics

Example fix

# before
out = impl.forward(q, k, v, meta)  # fa_ver=5 unsupported
# after: remove override / use supported version
out = impl.forward(q, k, v, meta)  # fa_ver resolves to 4

SUPPORTED_FA_VERS = {2, 3, 4}  # mirror the dispatch branches in flash_attn.forward
assert fa_ver in SUPPORTED_FA_VERS, f"fa_ver {fa_ver} unsupported"

try:
    out = impl.forward(q, k, v, meta)
except ValueError as e:
    if "flash attention version" in str(e):
        out = impl.forward(q, k, v, meta)  # without the ver override

When it happens

Trigger: Calling forward with a fa_ver not covered by the dispatch branches (e.g. an experimental integer set via config/env override, or set by version autodetection after upgrading flash-attn).

Common situations: Setting a custom attention version flag for a kernel this SGLang version predates; upgrading flash-attn so autodetection picks an unsupported major; hand-editing config with a bad ver integer.

Related errors

• Unsupported library: {transformers_or_diffusers}
• batching config schema_version must be 1
• This browser cannot encode H.264 MP4
• H.264 encoder did not return MP4 decoder config
• This browser does not support gzip stream decoding

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/25d2cc7a6109b411. Report an issue: GitHub.