sgl-project/sglang · error · ImportError

FlashAttention-4 CUTE is not available. Install flash-attn-4

Error message

FlashAttention-4 CUTE is not available. Install flash-attn-4 with its CUDA/CUTE dependencies, or run from a source tree where the vendored FA4 package is importable.

What it means

Same as the FA4 import guard: the sm120 FA4 varlen wrapper lazily imports the flash-attn-4 CUTE package and raises ImportError with the original failure chained if it is unavailable.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attention_v4_sm120.py:112

    sinks: Optional[torch.Tensor] = None,
    num_splits: int = 1,
    pack_gqa: Optional[bool] = None,
    score_mod: Optional[Callable] = None,
    aux_tensors: Optional[list] = None,
    q_descale: Optional[torch.Tensor] = None,
    k_descale: Optional[torch.Tensor] = None,
    v_descale: Optional[torch.Tensor] = None,
    sfq: Optional[torch.Tensor] = None,
    sfk: Optional[torch.Tensor] = None,
    sfv: Optional[torch.Tensor] = None,
    rel_bias: Optional[torch.Tensor] = None,
    rel_bias_prep_cache: Optional[dict] = None,
    return_softmax_lse: bool = False,
    out: Optional[torch.Tensor] = None,
    **_: object,
):
    if _flash_attn_varlen_func is None:  # pragma: no cover
        raise ImportError(
            "FlashAttention-4 CUTE is not available. Install flash-attn-4 with "
            "its CUDA/CUTE dependencies, or run from a source tree where the "
            "vendored FA4 package is importable."
        ) from _flash_attn_import_error

    _validate_out_contract(out)
    q, k, v, qv = [_maybe_contiguous(t) for t in (q, k, v, qv)]
    if qv is None and q.shape[-1] == 256 and k.shape[-1] == 256 and v.shape[-1] == 256:
        # The vendored hd256 kernel assumes dense Q/K/V strides.
        q, k, v = [t.contiguous() for t in (q, k, v)]
    q, qv, mla_head_padding = _pad_mla_q_heads(q, qv, v, pack_gqa)
    if qv is not None and num_splits < 1:
        # FA4 MLA does not implement split-KV; auto mode must use one split.
        num_splits = 1
    cu_seqlens_q, cu_seqlens_k = [
        _maybe_contiguous(t) for t in (cu_seqlens_q, cu_seqlens_k)
    ]
    seqused_q, seqused_k = [_maybe_contiguous(t) for t in (seqused_q, seqused_k)]

View on GitHub (pinned to 0132848349)

Solutions

1. Install flash-attn-4 with CUDA/CUTE deps or run from the sglang source tree
2. Inspect e.__cause__ for the underlying import failure
3. Sanity-check the import in the same Python env before launching the server

Example fix

# before
# ImportError at first attention call
# after
pip install flash-attn-4 --no-build-isolation
python -c 'import sglang.kernels.ops.attention.flash_attention_v4_sm120'

from sglang.kernels.ops.attention import flash_attention_v4_sm120 as fa\nassert fa._flash_attn_varlen_func is not None, 'flash-attn-4 not importable'

try:\n    out = fa.flash_attn_varlen_func(...)\nexcept ImportError as e:\n    log.error('FA4 unavailable: %s', e.__cause__); use_fa4 = False

When it happens

Trigger: Calling flash_attn_varlen_func / flash_attn_with_kvcache from flash_attention_v4_sm120 when the FA4 package import failed at module load.

Common situations: Missing or broken flash-attn-4 install; running an installed wheel without vendored FA4 sources; wrong environment/venv.

Related errors

• FlashAttention-4 CUTE is not available. Install flash-attn-4
• FA4 does not support updating KV cache in-place.
• FA4 path does not support rotary embedding.
• FA4 path does not support non-consecutive batch indices or l
• Can not import FA3 in sgl_kernel. Please check your installa

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/688f6f2a8623db62. Report an issue: GitHub.