sgl-project/sglang · error · ValueError

{name} must be a 1D int32 or int64 tensor

Error message

{name} must be a 1D int32 or int64 tensor

What it means

_packed_boundaries requires cu_seqlens tensors to be 1D and of dtype int32 or int64 — the NPU fused attention kernel accepts only these. A 2D tensor, a list, or another dtype (e.g. int16, float) fails this check.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py:31

from sglang.multimodal_gen.runtime.platforms import AttentionBackendEnum
from sglang.multimodal_gen.runtime.utils.logging_utils import init_logger

logger = init_logger(__name__)


def _packed_boundaries(
    cu_seqlens: torch.Tensor,
    cu_seqlens_host: Sequence[int] | None,
    total_tokens: int,
    name: str,
) -> tuple[int, ...]:
    if cu_seqlens is None:
        raise ValueError(f"{name} is required for NPU packed attention")
    if cu_seqlens.ndim != 1 or cu_seqlens.dtype not in (
        torch.int32,
        torch.int64,
    ):
        raise ValueError(f"{name} must be a 1D int32 or int64 tensor")
    if cu_seqlens_host is not None and len(cu_seqlens_host) != cu_seqlens.numel():
        raise ValueError(f"{name} and its host copy must have the same length")

    boundaries = tuple(
        int(value)
        for value in (
            cu_seqlens.tolist() if cu_seqlens_host is None else cu_seqlens_host
        )
    )
    if len(boundaries) < 2 or boundaries[0] != 0:
        raise ValueError(f"{name} must start with 0 and contain at least one sequence")
    if boundaries[-1] != total_tokens:
        raise ValueError(
            f"{name} must end at the packed token count {total_tokens}, "
            f"got {boundaries[-1]}"
        )
    if any(stop < start for start, stop in zip(boundaries[:-1], boundaries[1:])):
        raise ValueError(f"{name} must be non-decreasing")

View on GitHub (pinned to 0132848349)

Solutions

1. Convert: cu = torch.as_tensor(cu, dtype=torch.int32, device=q.device) and ensure it is 1D (squeeze a leading size-1 dim if present)
2. Build from lengths: cu = torch.nn.functional.pad(torch.tensor(lens).cumsum(0), (1,1)) then cast
3. Add an assertion before the call: assert cu.ndim == 1 and cu.dtype in (torch.int32, torch.int64)

Example fix

# before
cu_seqlens_q = [0, 5, 12]  # python list -> ndim fails
# after
cu_seqlens_q = torch.tensor([0, 5, 12], dtype=torch.int32, device=q.device)

def to_cu_seqlens(x, device) -> torch.Tensor:
    t = torch.as_tensor(x, dtype=torch.int32, device=device)
    assert t.ndim == 1
    return t

def is_valid_cu_seqlens(t) -> bool:
    return isinstance(t, torch.Tensor) and t.ndim == 1 and t.dtype in (torch.int32, torch.int64)

When it happens

Trigger: Calling fused_infer_attention_varlen with cu_seqlens passed as a Python list (no .ndim), a 2D tensor like shape [1, B+1], or a tensor of dtype bfloat16/float32/int16. Also triggered if a host-side list is passed where the tensor was expected.

Common situations: Callers constructing cu_seqlens with torch.tensor(lens).cumsum(0) without .to(torch.int32); code that passes numpy arrays or nested lists; porting code from another backend that accepted different dtypes.

Related errors

• {name} is required for NPU packed attention
• {name} and its host copy must have the same length
• {name} must start with 0 and contain at least one sequence
• {name} must end at the packed token count {total_tokens}, go
• {name} must be non-decreasing

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/be5e602b1bee61c5. Report an issue: GitHub.