{"record":{"id":"4b4ac95ab0eac0f2","repo":"sgl-project/sglang","slug":"flash-attention-currently-only-supported-for-compu","errorCode":null,"errorMessage":"Flash attention currently only supported for compute capability >= 80","messagePattern":"Flash attention currently only supported for compute capability >= 80","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/linear/lightning_attn.py","lineNumber":407,"sourceCode":"    tl.store(\n        O_block_ptr, qkv.to(O_block_ptr.dtype.element_ty), mask=q_index[:, None] < n\n    )\n\n\nclass _attention(torch.autograd.Function):\n\n    @staticmethod\n    def forward(ctx, q, k, v, s, kv_history):\n        # Forward pass of the lightning attention algorithm\n        q = q.contiguous()\n        k = k.contiguous()\n        v = v.contiguous()\n        s = s.contiguous()\n\n        # Check CUDA compute capability\n        capability = torch.cuda.get_device_capability()\n        if capability[0] < 8:\n            raise RuntimeError(\n                \"Flash attention currently only supported\",\n                \"for compute capability >= 80\",\n            )\n\n        # Get input dimensions\n        b, h, n, d = q.shape\n        e = v.shape[-1]\n\n        # Initialize output tensor\n        o = torch.empty((b, h, n, e), dtype=q.dtype, device=q.device)\n\n        # Set block sizes\n        BLOCK = 256\n        NUM_BLOCK = triton.cdiv(n, BLOCK)\n\n        CBLOCK = 32\n        NUM_CBLOCK = BLOCK // CBLOCK\n        assert BLOCK % CBLOCK == 0, \"BLOCK must be a multiple of CBLOCK\"","sourceCodeStart":389,"sourceCodeEnd":425,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/linear/lightning_attn.py#L389-L425","documentation":"Lightning attention's Triton/flash kernels require Ampere-or-newer tensor-core instructions; the forward explicitly checks torch.cuda.get_device_capability() and raises RuntimeError when the major version is below 8 (i.e. SM < 80, such as V100/T4).","triggerScenarios":"Calling lightning_attn forward (lightning_attn.py:forward) on a pre-Ampere GPU (compute capability 7.x or lower).","commonSituations":"Running or unit-testing lightning attention models on V100, GTX/RTX Turing, or older datacenter GPUs; defaulting to the lightning_attn backend on a heterogeneous cluster with older cards.","solutions":["Run on an SM80+ GPU (A100, A10, H100, L40S, RTX 30xx/40xx)","Select a non-flash attention backend (e.g. naive/triton path or a different attention implementation) for pre-Ampere devices","Gate the backend choice on torch.cuda.get_device_capability() at startup"],"exampleFix":"# before\nbackend = 'lightning_attn'  # crashes on V100\n# after\nmajor, _ = torch.cuda.get_device_capability()\nbackend = 'lightning_attn' if major >= 8 else 'triton'","handlingStrategy":"fallback","validationCode":"if torch.cuda.get_device_capability()[0] < 8:\n    attn_backend = 'naive'  # or any non-flash path\nelse:\n    attn_backend = 'lightning_attn'","typeGuard":null,"tryCatchPattern":"except RuntimeError as e: if 'compute capability' in str(e): use fallback attention backend","preventionTips":["Probe device capability at startup and select backends accordingly","Document GPU requirements for flash/lightning attention paths"],"tags":["cuda","gpu-capability","lightning-attn","hardware-unsupported"],"backgroundTag":"unsupported-gpu-compute-capability","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}