sgl-project/sglang · error · ValueError

flash_attn_varlen_func_op is out-only op; return_softmax_lse

Error message

flash_attn_varlen_func_op is out-only op; return_softmax_lse must be False. Use flash_attn_varlen_func_op_lse for (out, lse).

What it means

flash_attn_varlen_func_op is the out-only variant of the packed varlen flash-attention wrapper; passing return_softmax_lse=True is a misuse because the function cannot return the LSE. The wrapper rejects it to keep the return type honest.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/flash_attn.py:197

    causal: bool = False,
    qv: Optional[torch.Tensor] = None,
    q_descale: Optional[torch.Tensor] = None,
    k_descale: Optional[torch.Tensor] = None,
    v_descale: Optional[torch.Tensor] = None,
    window_size: Optional[List[int]] = None,
    attention_chunk: int = 0,
    softcap: float = 0.0,
    num_splits: int = 1,
    pack_gqa: Optional[bool] = None,
    sm_margin: int = 0,
    return_softmax_lse: bool = False,
    sinks: Optional[torch.Tensor] = None,
    ver: int = 4,
) -> torch.Tensor:
    if window_size is None:
        window_size = [-1, -1]
    if return_softmax_lse:
        raise ValueError(
            "flash_attn_varlen_func_op is out-only op; return_softmax_lse must be False. "
            "Use flash_attn_varlen_func_op_lse for (out, lse)."
        )
    return flash_attn_varlen_func(
        q,
        k,
        v,
        cu_seqlens_q=cu_seqlens_q,
        cu_seqlens_k=cu_seqlens_k,
        max_seqlen_q=max_seqlen_q,
        max_seqlen_k=max_seqlen_k,
        seqused_q=seqused_q,
        seqused_k=seqused_k,
        page_table=page_table,
        softmax_scale=softmax_scale,
        causal=causal,
        qv=qv,
        q_descale=q_descale,

View on GitHub (pinned to 0132848349)

Solutions

1. Switch the call to flash_attn_varlen_func_op_lse(...) and unpack (out, lse)
2. If only output is needed, pass return_softmax_lse=False (the default)

Example fix

# before
out = flash_attn_varlen_func_op(q, k, v, cu_q, cu_k, mq, mk, return_softmax_lse=True)
# after
out, lse = flash_attn_varlen_func_op_lse(q, k, v, cu_q, cu_k, mq, mk, return_softmax_lse=True)

assert not return_softmax_lse, "use flash_attn_varlen_func_op_lse for (out, lse)"

When it happens

Trigger: Calling flash_attn_varlen_func_op(..., return_softmax_lse=True); the check trips before any kernel dispatch.

Common situations: Code refactored from a single API that took a return_softmax_lse flag for both variants; adding LSE support (ring attention, logprob capture) by flipping the flag instead of switching functions.

Related errors

• flash_attn_varlen_func_op_lse is out+lse op; return_softmax_
• LSE tensor must be Float32
• cu_seqlens_q tensor must be Int32
• cu_seqlens_k tensor must be Int32
• Unexpected Ascend TND softmax LSE shape: expected {(q.shape[

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e78a859c300e0c7a. Report an issue: GitHub.