{"record":{"id":"1b6aae06de537168","repo":"sgl-project/sglang","slug":"flashattention-4-cute-is-not-available-install-fl","errorCode":null,"errorMessage":"FlashAttention-4 CUTE is not available. Install flash-attn-4 with its CUDA/CUTE dependencies, or run from a source tree where the vendored FA4 package is importable.","messagePattern":"FlashAttention-4 CUTE is not available\\. Install flash-attn-4 with its CUDA/CUTE dependencies, or run from a source tree where the vendored FA4 package is importable\\.","errorType":"validation","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attention_v4.py","lineNumber":132,"sourceCode":"    ] = None,  # legacy per-tensor FP8 descale scalar (fp8_e4m3/e5m2 KV)\n    k_descale: Optional[torch.Tensor] = None,  # legacy per-tensor FP8 descale scalar\n    v_descale: Optional[torch.Tensor] = None,  # legacy per-tensor FP8 descale scalar\n    sfq: Optional[\n        torch.Tensor\n    ] = None,  # MXFP8 UE8M0 per-32-elem block scales (block-scaled QK^T)\n    sfk: Optional[\n        torch.Tensor\n    ] = None,  # MXFP8 UE8M0 per-32-elem block scales (block-scaled QK^T)\n    sfv: Optional[\n        torch.Tensor\n    ] = None,  # MXFP8 UE8M0 per-32-elem block scales (in-kernel V dequant)\n    rel_bias: Optional[torch.Tensor] = None,\n    rel_bias_prep_cache: Optional[dict] = None,\n    return_softmax_lse: bool = False,\n    **_: object,\n):\n    if _flash_attn_varlen_func is None:  # pragma: no cover\n        raise ImportError(\n            \"FlashAttention-4 CUTE is not available. Install flash-attn-4 with \"\n            \"its CUDA/CUTE dependencies, or run from a source tree where the \"\n            \"vendored FA4 package is importable.\"\n        ) from _flash_attn_import_error\n\n    q, k, v, qv = [_maybe_contiguous(t) for t in (q, k, v, qv)]\n    if qv is None and q.shape[-1] == 256 and k.shape[-1] == 256 and v.shape[-1] == 256:\n        # The vendored hd256 kernel assumes dense Q/K/V strides.\n        # TODO: Remove this workaround after the FA4 in current environment includes\n        # https://github.com/Dao-AILab/flash-attention/pull/2670 (flash-attn-4 >= 4.0.0b20).\n        q, k, v = [t.contiguous() for t in (q, k, v)]\n    q, qv, mla_head_padding = _pad_mla_q_heads(q, qv, v, pack_gqa)\n    if qv is not None and num_splits < 1:\n        # FA4 MLA does not implement split-KV; auto mode must use one split.\n        num_splits = 1\n    cu_seqlens_q, cu_seqlens_k = [\n        _maybe_contiguous(t) for t in (cu_seqlens_q, cu_seqlens_k)\n    ]","sourceCodeStart":114,"sourceCodeEnd":150,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attention_v4.py#L114-L150","documentation":"The FA4 (FlashAttention-4 CUTE) varlen wrapper is a lazy import: if the flash-attn-4 package (or the vendored FA4 source tree) could not be imported, the wrapper raises ImportError chained to the original import error.","triggerScenarios":"Calling flash_attn_varlen_func (or flash_attn_with_kvcache which delegates to it) from flash_attention_v4 when _flash_attn_varlen_func is None because flash-attn-4 failed to import.","commonSituations":"Missing flash-attn-4 dependency; running an installed wheel without the vendored FA4 CUTE sources; partial/broken flash-attn install; wrong Python env.","solutions":["pip install the flash-attn-4 package (with CUDA/CUTE deps) or run from the sglang source tree containing the vendored FA4 package","Inspect the chained __cause__ import error to see which module/dependency failed","Verify the import works: python -c 'import flash_attn_4' (or the vendored path) before launching"],"exampleFix":"# before: ImportError at runtime\n# after\npip install flash-attn-4 --no-build-isolation\npython -c 'from sglang.kernels.ops.attention import flash_attention_v4'","handlingStrategy":"validation","validationCode":"from sglang.kernels.ops.attention import flash_attention_v4 as fa4\\nassert fa4._flash_attn_varlen_func is not None, 'flash-attn-4 not installed'","typeGuard":null,"tryCatchPattern":"try:\\n    out = fa4.flash_attn_varlen_func(...)\\nexcept ImportError as e:\\n    raise SystemExit(f'install flash-attn-4: {e.__cause__}') from e","preventionTips":["Verify FA4 import at startup, not first attention call","Keep flash-attn-4 pinned in requirements when using FA4"],"tags":["flash-attention","fa4","import-error","missing-dependency"],"backgroundTag":"optional-dependency-not-installed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}