sgl-project/sglang · error · ImportError

FlashAttention-4 CUTE is not available. Install flash-attn-4

Error message

FlashAttention-4 CUTE is not available. Install flash-attn-4 with its CUDA/CUTE dependencies, or run from a source tree where the vendored FA4 package is importable.

What it means

The FA4 (FlashAttention-4 CUTE) varlen wrapper is a lazy import: if the flash-attn-4 package (or the vendored FA4 source tree) could not be imported, the wrapper raises ImportError chained to the original import error.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attention_v4.py:132

    ] = None,  # legacy per-tensor FP8 descale scalar (fp8_e4m3/e5m2 KV)
    k_descale: Optional[torch.Tensor] = None,  # legacy per-tensor FP8 descale scalar
    v_descale: Optional[torch.Tensor] = None,  # legacy per-tensor FP8 descale scalar
    sfq: Optional[
        torch.Tensor
    ] = None,  # MXFP8 UE8M0 per-32-elem block scales (block-scaled QK^T)
    sfk: Optional[
        torch.Tensor
    ] = None,  # MXFP8 UE8M0 per-32-elem block scales (block-scaled QK^T)
    sfv: Optional[
        torch.Tensor
    ] = None,  # MXFP8 UE8M0 per-32-elem block scales (in-kernel V dequant)
    rel_bias: Optional[torch.Tensor] = None,
    rel_bias_prep_cache: Optional[dict] = None,
    return_softmax_lse: bool = False,
    **_: object,
):
    if _flash_attn_varlen_func is None:  # pragma: no cover
        raise ImportError(
            "FlashAttention-4 CUTE is not available. Install flash-attn-4 with "
            "its CUDA/CUTE dependencies, or run from a source tree where the "
            "vendored FA4 package is importable."
        ) from _flash_attn_import_error

    q, k, v, qv = [_maybe_contiguous(t) for t in (q, k, v, qv)]
    if qv is None and q.shape[-1] == 256 and k.shape[-1] == 256 and v.shape[-1] == 256:
        # The vendored hd256 kernel assumes dense Q/K/V strides.
        # TODO: Remove this workaround after the FA4 in current environment includes
        # https://github.com/Dao-AILab/flash-attention/pull/2670 (flash-attn-4 >= 4.0.0b20).
        q, k, v = [t.contiguous() for t in (q, k, v)]
    q, qv, mla_head_padding = _pad_mla_q_heads(q, qv, v, pack_gqa)
    if qv is not None and num_splits < 1:
        # FA4 MLA does not implement split-KV; auto mode must use one split.
        num_splits = 1
    cu_seqlens_q, cu_seqlens_k = [
        _maybe_contiguous(t) for t in (cu_seqlens_q, cu_seqlens_k)
    ]

View on GitHub (pinned to 0132848349)

Solutions

1. pip install the flash-attn-4 package (with CUDA/CUTE deps) or run from the sglang source tree containing the vendored FA4 package
2. Inspect the chained __cause__ import error to see which module/dependency failed
3. Verify the import works: python -c 'import flash_attn_4' (or the vendored path) before launching

Example fix

# before: ImportError at runtime
# after
pip install flash-attn-4 --no-build-isolation
python -c 'from sglang.kernels.ops.attention import flash_attention_v4'

from sglang.kernels.ops.attention import flash_attention_v4 as fa4\nassert fa4._flash_attn_varlen_func is not None, 'flash-attn-4 not installed'

try:\n    out = fa4.flash_attn_varlen_func(...)\nexcept ImportError as e:\n    raise SystemExit(f'install flash-attn-4: {e.__cause__}') from e

When it happens

Trigger: Calling flash_attn_varlen_func (or flash_attn_with_kvcache which delegates to it) from flash_attention_v4 when _flash_attn_varlen_func is None because flash-attn-4 failed to import.

Common situations: Missing flash-attn-4 dependency; running an installed wheel without the vendored FA4 CUTE sources; partial/broken flash-attn install; wrong Python env.

Related errors

• FlashAttention-4 CUTE is not available. Install flash-attn-4
• Can not import FA3 in sgl_kernel. Please check your installa
• FA4 does not support updating KV cache in-place.
• FA4 path does not support rotary embedding.
• FA4 path does not support non-consecutive batch indices or l

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1b6aae06de537168. Report an issue: GitHub.