sgl-project/sglang · error · ValueError

Invalid version: {self.fa_impl_ver=}

Error message

Invalid version: {self.fa_impl_ver=}

What it means

FlashAttentionBackend dispatches among FA2/FA3/FA4 implementations based on self.fa_impl_ver; any unrecognized value falls into the else branch and raises. The version is auto-detected from installed flash-attn/flash-attn-3/cute packages or forced via env/args.

Source

Thrown at python/sglang/srt/layers/attention/flashattention_backend.py:300

                )
            else:
                from sglang.kernels.ops.attention.flash_attention_v4 import (
                    flash_attn_varlen_func,
                    flash_attn_with_kvcache,
                )

                self._get_fa_runtime_policy = None

            self._get_scheduler_metadata = None
            if get_exec().deterministic.enable_deterministic_inference:
                # Must precede the first kernel compile.
                from sglang.kernels.ops.attention.flash_attn.cute.batch_invariance import (
                    set_batch_invariant,
                )

                set_batch_invariant(True)
        else:
            raise ValueError(f"Invalid version: {self.fa_impl_ver=}")

        self.flash_attn_varlen_func = flash_attn_varlen_func
        self.flash_attn_with_kvcache = flash_attn_with_kvcache

        # Store head info for precomputing FA3 scheduler metadata
        self.head_dim = model_runner.model_config.head_dim
        self.num_attention_heads = (
            model_runner.model_config.hf_text_config.num_attention_heads
            // model_runner.ps.tp_size
        )
        self.num_kv_heads = model_runner.model_config.get_num_kv_heads(
            model_runner.ps.tp_size
        )
        _softcapping = getattr(
            model_runner.model_config.hf_text_config, "attn_logit_softcapping", None
        )
        self.has_softcap = _softcapping is not None and _softcapping > 0.0

View on GitHub (pinned to 0132848349)

Solutions

1. Unset/fix any override such as SGLANG_FLASHATTENTION_VERSION so it is 2, 3, or 4
2. Reinstall a compatible flash-attn package matching the SGLang version requirement
3. Let auto-detection pick the version (remove manual pinning) and retry startup

Example fix

# before
export SGLANG_FLASHATTENTION_VERSION=5
# after
unset SGLANG_FLASHATTENTION_VERSION  # or set to 2/3/4

ver = int(os.environ.get('SGLANG_FLASHATTENTION_VERSION', '0') or 0)
assert ver in (0, 2, 3, 4), f'bad FA version override {ver}'

try:
    backend = FlashAttentionBackend(...)
except ValueError as e:
    if 'Invalid version' in str(e):
        os.environ.pop('SGLANG_FLASHATTENTION_VERSION', None)
        backend = FlashAttentionBackend(...)  # re-init with autodetect

When it happens

Trigger: Constructing FlashAttentionBackend when fa_impl_ver was set to something other than 2, 3 or 4 — e.g. manually overriding the version env var, or an auto-detection path that returned an invalid sentinel after a partial/broken flash-attn install.

Common situations: Mixing flash-attn versions (pip flash-attn + flash-attn-3 + cute-dsl combos), stale SGLANG env overrides like SGLANG_FLASHATTENTION_VERSION=5, or upgrading SGLang against an unsupported flash-attn build.

Related errors

• Can not import FA3 in sgl_kernel. Please check your installa
• v_cache must be provided
• k_cache can only be None when only_qv=True
• q can only be None when only_qv=True
• q must be provided unless qv is provided with only_qv=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/520bbd6c028b2387. Report an issue: GitHub.