sgl-project/sglang · error · ValueError

{name} must be non-decreasing

Error message

{name} must be non-decreasing

What it means

_packed_boundaries requires the cumulative boundary sequence to be non-decreasing (each stop >= start). Since cu_seqlens are cumulative sums of nonnegative lengths, any decrease indicates corrupted or misordered data — usually negative sequence lengths or a bad host copy.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py:49

        raise ValueError(f"{name} must be a 1D int32 or int64 tensor")
    if cu_seqlens_host is not None and len(cu_seqlens_host) != cu_seqlens.numel():
        raise ValueError(f"{name} and its host copy must have the same length")

    boundaries = tuple(
        int(value)
        for value in (
            cu_seqlens.tolist() if cu_seqlens_host is None else cu_seqlens_host
        )
    )
    if len(boundaries) < 2 or boundaries[0] != 0:
        raise ValueError(f"{name} must start with 0 and contain at least one sequence")
    if boundaries[-1] != total_tokens:
        raise ValueError(
            f"{name} must end at the packed token count {total_tokens}, "
            f"got {boundaries[-1]}"
        )
    if any(stop < start for start, stop in zip(boundaries[:-1], boundaries[1:])):
        raise ValueError(f"{name} must be non-decreasing")
    return boundaries


def fused_infer_attention_varlen(
    q: torch.Tensor,
    k: torch.Tensor,
    v: torch.Tensor,
    cu_seqlens_q: torch.Tensor,
    cu_seqlens_k: torch.Tensor,
    *,
    cu_seqlens_q_host: Sequence[int] | None = None,
    cu_seqlens_k_host: Sequence[int] | None = None,
    softmax_scale: float | None = None,
    return_softmax_lse: bool = False,
) -> torch.Tensor | tuple[torch.Tensor, torch.Tensor]:
    tensors = {"q": q, "k": k, "v": v}
    invalid_layouts = [name for name, tensor in tensors.items() if tensor.ndim != 3]
    if invalid_layouts:

View on GitHub (pinned to 0132848349)

Solutions

1. Always construct boundaries via torch.tensor(lens, dtype=torch.int32).cumsum(0) padded with a leading 0 rather than manual arithmetic
2. Validate lengths are nonnegative before cumsum: assert all(l >= 0 for l in lens)
3. If using cu_seqlens_host, verify it matches a freshly computed cumsum of the current lengths

Example fix

# before
cu = torch.tensor([0, 10, 7, 20], dtype=torch.int32)  # decreases
# after
lens = [10, 0, 13]
cu = torch.nn.functional.pad(torch.tensor(lens, dtype=torch.int32).cumsum(0), (1, 1), value=0)  # [0,10,10,23,23]... use [0]+cumsum

b = cu.tolist()
assert all(b[i] <= b[i+1] for i in range(len(b)-1)), "cu_seqlens must be non-decreasing"

When it happens

Trigger: Passing cu_seqlens values that decrease anywhere, e.g. [0, 10, 7, 20], or a cu_seqlens_host list built with negative/garbage lengths (e.g. mismatched int parsing).

Common situations: Arithmetic bugs when computing boundaries manually instead of cumsum; a corrupted host copy from a previous batch; sign errors after rebase arithmetic.

Related errors

• {name} must start with 0 and contain at least one sequence
• {name} is required for NPU packed attention
• {name} and its host copy must have the same length
• {name} must end at the packed token count {total_tokens}, go
• cu_seqlens_q and cu_seqlens_k must describe the same batch

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b349949cbf153515. Report an issue: GitHub.