{"record":{"id":"534caea3de719a66","repo":"sgl-project/sglang","slug":"flash-attn-at-sgl-kernel-is-only-supported-on-sm90-534cae","errorCode":null,"errorMessage":"flash_attn at sgl-kernel is only supported on sm90 and above","messagePattern":"flash_attn at sgl-kernel is only supported on sm90 and above","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attention_v3.py","lineNumber":138,"sourceCode":"    k_descale: Optional[torch.Tensor] = None,\n    v_descale: Optional[torch.Tensor] = None,\n    softmax_scale=None,\n    causal=False,\n    window_size=(-1, -1),  # -1 means infinite context window\n    attention_chunk: Optional[int] = None,\n    softcap=0.0,  # 0.0 means deactivated\n    rotary_interleaved=True,\n    scheduler_metadata=None,\n    num_splits=0,  # Can be tuned for speed\n    pack_gqa=None,  # Can be tuned for speed\n    only_qv=False,  # Skip K matmul when qk rope dim is 0 (requires qv)\n    sm_margin=0,  # Can be tuned if some SMs are used for communication\n    return_softmax_lse=False,\n    sinks=None,\n    out=None,\n):\n    if not _is_fa3_supported():\n        raise NotImplementedError(\n            \"flash_attn at sgl-kernel is only supported on sm90 and above\"\n        )\n\n    # When only_qv=True the caller may pass k_cache=None (synthetic K is\n    # allocated inside the sgl-kernel wrapper). Skip the stride check in that\n    # case so the rope=0 path doesn't trip the assertion.\n    if k_cache is not None:\n        assert k_cache.stride(-1) == 1, \"k_cache must have contiguous last dimension\"\n    assert v_cache.stride(-1) == 1, \"v_cache must have contiguous last dimension\"\n\n    return _call_fa3_kernel(\n        _load_fa3_kernels()[\"flash_attn_with_kvcache\"],\n        q,\n        k_cache,\n        v_cache,\n        k,\n        v,\n        qv,","sourceCodeStart":120,"sourceCodeEnd":156,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attention_v3.py#L120-L156","documentation":"The sgl-kernel FA3 path (flash_attention_v3.flash_attn_with_kvcache) only runs on NVIDIA sm90+ (Hopper and later). On older GPUs _is_fa3_supported() is false and it raises NotImplementedError immediately.","triggerScenarios":"Calling flash_attn_with_kvcache from flash_attention_v3 on a GPU with compute capability < 9.0 (Ampere sm80/sm86, Ada sm89, Turing).","commonSituations":"Running a config that selects FA3 on A100/30xx/40xx/T4 hardware; default server args choosing fa3 backend on non-Hopper machines.","solutions":["Use FA2 backend or another attention backend supported on your GPU","Set the server attention backend explicitly (e.g. --attention-backend fa2/flashinfer/triton)","Upgrade to an sm90+ GPU (H100/H200/B200) if FA3 features are required"],"exampleFix":"# before\npython -m sglang.launch_server --model ... --attention-backend fa3  # on A100\n# after\npython -m sglang.launch_server --model ... --attention-backend fa2","handlingStrategy":"fallback","validationCode":"import torch\\nmajor, _ = torch.cuda.get_device_capability()\\nif major < 9:\\n    ver = 2  # FA3 needs sm90+; fall back to FA2","typeGuard":"def fa3_supported() -> bool:\\n    return torch.cuda.get_device_capability()[0] >= 9","tryCatchPattern":"try:\\n    out = fa3.flash_attn_with_kvcache(...)\\nexcept NotImplementedError:\\n    out = fa2.flash_attn_with_kvcache(...)","preventionTips":["Check torch.cuda.get_device_capability() at startup","Pin attention backend per hardware in deployment config"],"tags":["flash-attention","fa3","gpu-compatibility","sm90","hardware-unsupported"],"backgroundTag":"cuda-compute-capability-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}