sgl-project/sglang · error · ValueError

{name} is required for NPU packed attention

Error message

{name} is required for NPU packed attention

What it means

_packed_boundaries validates the cumulative-seqlens tensors required by NPU (Ascend) packed/varlen attention. cu_seqlens_q (and cu_seqlens_k) must not be None; they encode per-sequence token boundaries as a 1D tensor starting at 0. Passing None means the caller skipped required inputs.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py:26

    AttentionBackend,
    AttentionImpl,
    AttentionMetadata,
    AttentionMetadataBuilder,
)
from sglang.multimodal_gen.runtime.platforms import AttentionBackendEnum
from sglang.multimodal_gen.runtime.utils.logging_utils import init_logger

logger = init_logger(__name__)


def _packed_boundaries(
    cu_seqlens: torch.Tensor,
    cu_seqlens_host: Sequence[int] | None,
    total_tokens: int,
    name: str,
) -> tuple[int, ...]:
    if cu_seqlens is None:
        raise ValueError(f"{name} is required for NPU packed attention")
    if cu_seqlens.ndim != 1 or cu_seqlens.dtype not in (
        torch.int32,
        torch.int64,
    ):
        raise ValueError(f"{name} must be a 1D int32 or int64 tensor")
    if cu_seqlens_host is not None and len(cu_seqlens_host) != cu_seqlens.numel():
        raise ValueError(f"{name} and its host copy must have the same length")

    boundaries = tuple(
        int(value)
        for value in (
            cu_seqlens.tolist() if cu_seqlens_host is None else cu_seqlens_host
        )
    )
    if len(boundaries) < 2 or boundaries[0] != 0:
        raise ValueError(f"{name} must start with 0 and contain at least one sequence")
    if boundaries[-1] != total_tokens:
        raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

1. Always build and pass cu_seqlens_q and cu_seqlens_k as int32/int64 tensors: [0, len0, len0+len1, ...]
2. Audit call sites in forward_varlen and forward_ring_kv_chunk to ensure neither argument is dropped
3. If sequences are uniform you still need cu_seqlens (repeat of cumsum), not None

Example fix

# before
out = fused_infer_attention_varlen(q, k, v, cu_seqlens_q=None, cu_seqlens_k=None)
# after
cu_q = torch.tensor([0, 5, 12, 20], dtype=torch.int32, device=q.device)
cu_k = torch.tensor([0, 9, 21, 34], dtype=torch.int32, device=k.device)
out = fused_infer_attention_varlen(q, k, v, cu_seqlens_q=cu_q, cu_seqlens_k=cu_k)

assert cu_seqlens_q is not None and cu_seqlens_k is not None, "cu_seqlens required for varlen NPU attention"

def has_cu_seqlens(*args: torch.Tensor | None) -> bool:
    return all(t is not None for t in args)

When it happens

Trigger: Calling fused_infer_attention_varlen (directly or via forward_varlen/forward_ring_kv_chunk) with cu_seqlens_q or cu_seqlens_k set to None; name is substituted with 'cu_seqlens_q' or 'cu_seqlens_k'.

Common situations: A varlen forward path that only materializes cu_seqlens in some branches (e.g. batch-of-one shortcut skips building them); refactoring that passes the host copy but not the device tensor, or vice versa.

Related errors

• {name} must start with 0 and contain at least one sequence
• {name} must end at the packed token count {total_tokens}, go
• {name} must be non-decreasing
• {name} must be a 1D int32 or int64 tensor
• {name} and its host copy must have the same length

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bf3b7d057ca152b0. Report an issue: GitHub.