sgl-project/sglang · error · ValueError

flash_attn_varlen_func_op_lse is out+lse op; return_softmax_

Error message

flash_attn_varlen_func_op_lse is out+lse op; return_softmax_lse must be True. Use flash_attn_varlen_func_op for out-only.

What it means

flash_attn_varlen_func_op_lse always returns a tuple (output, softmax_lse); calling it with return_softmax_lse=False is contradictory, so it raises and points you to the out-only flash_attn_varlen_func_op.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/flash_attn.py:261

    causal: bool = False,
    qv: Optional[torch.Tensor] = None,
    q_descale: Optional[torch.Tensor] = None,
    k_descale: Optional[torch.Tensor] = None,
    v_descale: Optional[torch.Tensor] = None,
    window_size: Optional[List[int]] = None,
    attention_chunk: int = 0,
    softcap: float = 0.0,
    num_splits: int = 1,
    pack_gqa: Optional[bool] = None,
    sm_margin: int = 0,
    return_softmax_lse: bool = True,
    sinks: Optional[torch.Tensor] = None,
    ver: int = 4,
) -> Tuple[torch.Tensor, torch.Tensor]:
    if window_size is None:
        window_size = [-1, -1]
    if not return_softmax_lse:
        raise ValueError(
            "flash_attn_varlen_func_op_lse is out+lse op; return_softmax_lse must be True. "
            "Use flash_attn_varlen_func_op for out-only."
        )
    return flash_attn_varlen_func(
        q,
        k,
        v,
        cu_seqlens_q=cu_seqlens_q,
        cu_seqlens_k=cu_seqlens_k,
        max_seqlen_q=max_seqlen_q,
        max_seqlen_k=max_seqlen_k,
        seqused_q=seqused_q,
        seqused_k=seqused_k,
        page_table=page_table,
        softmax_scale=softmax_scale,
        causal=causal,
        qv=qv,
        q_descale=q_descale,

View on GitHub (pinned to 0132848349)

Solutions

1. Use flash_attn_varlen_func_op(...) for out-only results
2. Or pass return_softmax_lse=True to _op_lse and unpack the 2-tuple

Example fix

# before
out = flash_attn_varlen_func_op_lse(q, k, v, cu_q, cu_k, mq, mk, return_softmax_lse=False)
# after
out = flash_attn_varlen_func_op(q, k, v, cu_q, cu_k, mq, mk, return_softmax_lse=False)

assert return_softmax_lse is True, "_op_lse always returns (out, lse); use _op for out-only"

When it happens

Trigger: Calling flash_attn_varlen_func_op_lse(..., return_softmax_lse=False).

Common situations: Copy-paste between the two variants during refactor; a shared kwargs dict defaulting return_softmax_lse=False while the call was switched to the _lse function.

Related errors

• flash_attn_varlen_func_op is out-only op; return_softmax_lse
• LSE tensor must be Float32
• cu_seqlens_q tensor must be Int32
• cu_seqlens_k tensor must be Int32
• Unexpected Ascend TND softmax LSE shape: expected {(q.shape[

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/846b3f4a1b037004. Report an issue: GitHub.