sgl-project/sglang · error · ValueError

LSE partial tensor must have 3 or 4 dimensions: (num_splits,

Error message

LSE partial tensor must have 3 or 4 dimensions: (num_splits, batch, seqlen, nheads) or (num_splits, total_q, nheads)

What it means

The partial LSE tensor must be 3D (num_splits, batch, nheads) batched or 4D (num_splits, total_q, nheads) varlen, matching the layout of the O partials.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd_combine.py:235

        stream: cuda.CUstream = None,
    ):
        # Type checking
        if const_expr(not (mO_partial.element_type == self.dtype_partial)):
            raise TypeError("O partial tensor must match dtype_partial")
        if const_expr(not (mO.element_type == self.dtype)):
            raise TypeError("O tensor must match dtype")
        if const_expr(mLSE_partial.element_type not in [Float32]):
            raise TypeError("LSE partial tensor must be Float32")
        if const_expr(mLSE is not None and mLSE.element_type not in [Float32]):
            raise TypeError("LSE tensor must be Float32")

        # Shape validation - input tensors are in user format, need to be converted to kernel format
        if const_expr(len(mO_partial.shape) not in [4, 5]):
            raise ValueError(
                "O partial tensor must have 4 or 5 dimensions: (num_splits, batch, seqlen, nheads, headdim) or (num_splits, total_q, nheads, headdim)"
            )
        if const_expr(len(mLSE_partial.shape) not in [3, 4]):
            raise ValueError(
                "LSE partial tensor must have 3 or 4 dimensions: (num_splits, batch, seqlen, nheads) or (num_splits, total_q, nheads)"
            )
        if const_expr(len(mO.shape) not in [3, 4]):
            raise ValueError(
                "O tensor must have 3 or 4 dimensions: (batch, seqlen, nheads, headdim) or (total_q, nheads, headdim)"
            )
        if const_expr(mLSE is not None and len(mLSE.shape) not in [2, 3]):
            raise ValueError(
                "LSE tensor must have 2 or 3 dimensions: (batch, seqlen, nheads) or (total_q, nheads)"
            )

        mO_partial, mO = [assume_tensor_aligned(t) for t in (mO_partial, mO)]
        # (num_splits, b, seqlen, h, d) -> (seqlen, d, num_splits, h, b)
        # or (num_splits, total_q, h, d) -> (total_q, d, num_splits, h)
        O_partial_layout_transpose = (
            [2, 4, 0, 3, 1] if const_expr(cu_seqlens is None) else [1, 3, 0, 2]
        )
        # (b, seqlen, h, d) -> (seqlen, d, h, b) or (total_q, h, d) -> (total_q, d, h)

View on GitHub (pinned to 0132848349)

Solutions

Match mLSE_partial rank to mO_partial: 4D O partials -> 3D LSE partials (num_splits,b,h); 5D O partials -> 4D (num_splits,total_q,h)
Verify seqlen/total_q axis matches the O partial tensor

Example fix

// before
mLSE_partial = lse.view(num_splits, -1, h)  # wrong for varlen
// after
mLSE_partial = lse.view(num_splits, total_q, h)

Defensive patterns

Strategy: validation

Validate before calling

assert mLSE_partial.dim() in (3, 4)
assert mLSE_partial.dim() == mO_partial.dim() - 1

Type guard

def lse_partial_ok(lse, o_partial) -> bool:
    return lse.dim() in (3, 4) and lse.dim() == o_partial.dim() - 1

Prevention

Derive LSE shape from the O partial shape programmatically instead of hand-writing it

When it happens

Trigger: Passing mLSE_partial with rank other than 3 or 4, or a layout inconsistent with the O partials (e.g. 2D LSE for 4D O partials).

Common situations: Computing LSE in a transposed shape (nheads, batch) then flattening; mismatched batched vs varlen call styles between O and LSE.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d099ccda997b2593. Report an issue: GitHub.