sgl-project/sglang · critical · RuntimeError

FlashAttention did not return the softmax LSE required by ri

Error message

FlashAttention did not return the softmax LSE required by ring attention

What it means

Ring attention needs the softmax LSE to merge partial results across KV chunks. forward_ring_kv_chunk calls flash-attention with return_softmax_lse=True and asserts the result is a tuple; if the dispatched kernel/wrapper path returns a bare tensor, merging would be impossible so it raises RuntimeError.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/flash_attn.py:503

        )
        cu_seqlens_k = torch.tensor(
            [0, key.shape[0]], dtype=torch.int32, device=key.device
        )
        result = flash_attn_varlen_func(
            query,
            key,
            value,
            cu_seqlens_q=cu_seqlens_q,
            cu_seqlens_k=cu_seqlens_k,
            max_seqlen_q=query.shape[0],
            max_seqlen_k=key.shape[0],
            softmax_scale=self.softmax_scale,
            causal=False,
            ver=fa_ver,
            return_softmax_lse=True,
        )
        if not isinstance(result, tuple):
            raise RuntimeError(
                "FlashAttention did not return the softmax LSE required by ring "
                "attention"
            )
        output, softmax_lse, *_ = result
        return output, softmax_lse

View on GitHub (pinned to 0132848349)

Solutions

Verify the call path uses a wrapper variant that honors return_softmax_lse=True and returns a tuple
Pin/upgrade the flash-attn integration so the dispatched version returns the LSE
Fall back to non-ring attention mode until fixed

Example fix

# before
result = flash_attn_varlen_func_op(q, k, v, ..., return_softmax_lse=True)  # tensor, not tuple -> RuntimeError downstream
# after
result = flash_attn_varlen_func_op_lse(q, k, v, ..., return_softmax_lse=True)  # (out, lse, ...)

Defensive patterns

Strategy: try-catch

Validate before calling

Ensure the wrapper invoked is flash_attn_varlen_func_op_lse with return_softmax_lse=True before entering forward_ring_kv_chunk.

Type guard

def returns_lse(result) -> bool:
    return isinstance(result, tuple) and len(result) >= 2

Try / catch

try:
    out, lse = impl.forward_ring_kv_chunk(q, kc, vc)
except RuntimeError as e:
    if "softmax LSE" in str(e):
        switch_to_non_ring_attention()

Prevention

Add unit tests asserting the ring path returns (out, lse)
Pin the flash-attn integration version

When it happens

Trigger: Calling forward_ring_kv_chunk when the underlying flash_attn_varlen_func dispatch returns only the output tensor (not (out, lse, ...)) despite return_softmax_lse=True.

Common situations: A wrapper split into _op/_op_lse variants breaking the tuple contract; a new fa_ver branch that forgets to propagate return_softmax_lse; flash-attn version mismatch dropping the LSE.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/652eb6793e3e3702. Report an issue: GitHub.