{"record":{"id":"652eb6793e3e3702","repo":"sgl-project/sglang","slug":"flashattention-did-not-return-the-softmax-lse-requ","errorCode":null,"errorMessage":"FlashAttention did not return the softmax LSE required by ring attention","messagePattern":"FlashAttention did not return the softmax LSE required by ring attention","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/layers/attention/backends/flash_attn.py","lineNumber":503,"sourceCode":"        )\n        cu_seqlens_k = torch.tensor(\n            [0, key.shape[0]], dtype=torch.int32, device=key.device\n        )\n        result = flash_attn_varlen_func(\n            query,\n            key,\n            value,\n            cu_seqlens_q=cu_seqlens_q,\n            cu_seqlens_k=cu_seqlens_k,\n            max_seqlen_q=query.shape[0],\n            max_seqlen_k=key.shape[0],\n            softmax_scale=self.softmax_scale,\n            causal=False,\n            ver=fa_ver,\n            return_softmax_lse=True,\n        )\n        if not isinstance(result, tuple):\n            raise RuntimeError(\n                \"FlashAttention did not return the softmax LSE required by ring \"\n                \"attention\"\n            )\n        output, softmax_lse, *_ = result\n        return output, softmax_lse\n","sourceCodeStart":485,"sourceCodeEnd":509,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/attention/backends/flash_attn.py#L485-L509","documentation":"Ring attention needs the softmax LSE to merge partial results across KV chunks. forward_ring_kv_chunk calls flash-attention with return_softmax_lse=True and asserts the result is a tuple; if the dispatched kernel/wrapper path returns a bare tensor, merging would be impossible so it raises RuntimeError.","triggerScenarios":"Calling forward_ring_kv_chunk when the underlying flash_attn_varlen_func dispatch returns only the output tensor (not (out, lse, ...)) despite return_softmax_lse=True.","commonSituations":"A wrapper split into _op/_op_lse variants breaking the tuple contract; a new fa_ver branch that forgets to propagate return_softmax_lse; flash-attn version mismatch dropping the LSE.","solutions":["Verify the call path uses a wrapper variant that honors return_softmax_lse=True and returns a tuple","Pin/upgrade the flash-attn integration so the dispatched version returns the LSE","Fall back to non-ring attention mode until fixed"],"exampleFix":"# before\nresult = flash_attn_varlen_func_op(q, k, v, ..., return_softmax_lse=True)  # tensor, not tuple -> RuntimeError downstream\n# after\nresult = flash_attn_varlen_func_op_lse(q, k, v, ..., return_softmax_lse=True)  # (out, lse, ...)","handlingStrategy":"try-catch","validationCode":"Ensure the wrapper invoked is flash_attn_varlen_func_op_lse with return_softmax_lse=True before entering forward_ring_kv_chunk.","typeGuard":"def returns_lse(result) -> bool:\n    return isinstance(result, tuple) and len(result) >= 2","tryCatchPattern":"try:\n    out, lse = impl.forward_ring_kv_chunk(q, kc, vc)\nexcept RuntimeError as e:\n    if \"softmax LSE\" in str(e):\n        switch_to_non_ring_attention()","preventionTips":["Add unit tests asserting the ring path returns (out, lse)","Pin the flash-attn integration version"],"tags":["flash-attention","ring-attention","lse","contract-violation"],"backgroundTag":"unexpected-return-type","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}