{"record":{"id":"8930f3f85994845f","repo":"sgl-project/sglang","slug":"can-not-import-fa3-in-sgl-kernel-please-check-you","errorCode":null,"errorMessage":"Can not import FA3 in sgl_kernel. Please check your installation.","messagePattern":"Can not import FA3 in sgl_kernel\\. Please check your installation\\.","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"critical","filePath":"python/sglang/kernels/aot/python/sgl_kernel/flash_attn.py","lineNumber":10,"sourceCode":"from functools import lru_cache\nfrom typing import Optional, Union\n\nimport torch\nfrom sgl_kernel.debug_utils import maybe_wrap_debug_kernel\n\ntry:\n    from sgl_kernel import flash_ops\nexcept:\n    raise ImportError(\n        \"Can not import FA3 in sgl_kernel. Please check your installation.\"\n    )\n\n\n@lru_cache(maxsize=1)\ndef is_fa3_supported(device=None) -> bool:\n    #  There some fa3 FYI\n    #  FA3 can fail without a enough shared memory for a some shapes, such as higher\n    #  hidden_dim or some special cases.\n    #  Right now, fa3 is supported for sm80/sm87 and sm86/sm89. The main different\n    #  Between sm80/sm87 and sm86/sm89 is the shared memory size. you can follow the link below for more information\n    #  https://docs.nvidia.com/cuda/cuda-c-programming-guide/#shared-memory-8-x\n    #  And for sgl-kernel right now, we can build fa3 on sm80/sm86/sm89/sm90a.\n    #  That means if you use A100/A*0/L20/L40/L40s/4090 you can use fa3.\n    return (torch.version.cuda >= \"12.3\") and (\n        torch.cuda.get_device_capability(device)[0] == 9\n        or torch.cuda.get_device_capability(device)[0] == 8\n    )","sourceCodeStart":1,"sourceCodeEnd":28,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/aot/python/sgl_kernel/flash_attn.py#L1-L28","documentation":"Module-level import guard in sgl_kernel/flash_attn.py: if `from sgl_kernel import flash_ops` fails for any reason, the module raises ImportError claiming FA3 cannot be imported. The bare except means the real cause (missing CUDA extension, libtorch ABI mismatch, missing libflashinfer/sgl shared libs) is hidden inside the chained exception.","triggerScenarios":"importing sgl_kernel.flash_attn when flash_ops (the compiled FA3 ops) failed to load — broken sgl-kernel wheel, CUDA extension not built for your torch/CUDA version, missing LD_LIBRARY_PATH for bundled .so files; any code path importing flash_attn (FA3 backend init, is_fa3_supported()).","commonSituations":"sgl-kernel version mismatched with installed torch; pip installed a wheel built for a different CUDA version; container missing GPU libs; upgrading torch without upgrading sgl-kernel.","solutions":["Reinstall sgl-kernel matching your torch/CUDA: `pip install -U sgl-kernel --force-reinstall` (pick the CUDA-specific wheel index if needed).","Check `python -c \"import sgl_kernel\"` to surface the real underlying error (the bare except hides it).","Verify torch version and CUDA runtime match the wheel's build (see sgl-kernel release notes for the torch/CUDA matrix).","Ensure LD_LIBRARY_PATH includes the sgl_kernel package dir if libs fail to resolve."],"exampleFix":"# before\npip install sgl-kernel  # may pull mismatched torch build\n# after\npip install sgl-kernel --force-reinstall --no-deps \\\n  --index-url https://flashinfer.ai/whl/cu124/torch2.5  # match your env","handlingStrategy":"try-catch","validationCode":"try:\n    import sgl_kernel.flash_ops  # noqa: F401\n    fa3_available = True\nexcept Exception:\n    fa3_available = False","typeGuard":"def fa3_available() -> bool:\n    try:\n        import sgl_kernel.flash_ops  # noqa: F401\n        return True\n    except Exception:\n        return False","tryCatchPattern":"try:\n    from sgl_kernel.flash_attn import flash_attn_with_kvcache\nexcept ImportError as e:\n    if \"FA3\" in str(e):\n        flash_attn_with_kvcache = None  # select triton/flashinfer backend instead","preventionTips":["Pin sgl-kernel to the wheel built for your exact torch+CUDA version.","Verify `import sgl_kernel` in environment setup scripts before long jobs.","Provide an attention-backend fallback (triton, flashinfer) in serving configs."],"tags":["sglang","flash-attention","import-error","native-extension","environment"],"backgroundTag":"native-extension-import-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}