{"record":{"id":"be5e602b1bee61c5","repo":"sgl-project/sglang","slug":"name-must-be-a-1d-int32-or-int64-tensor","errorCode":null,"errorMessage":"{name} must be a 1D int32 or int64 tensor","messagePattern":"(.+?) must be a 1D int32 or int64 tensor","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py","lineNumber":31,"sourceCode":"from sglang.multimodal_gen.runtime.platforms import AttentionBackendEnum\nfrom sglang.multimodal_gen.runtime.utils.logging_utils import init_logger\n\nlogger = init_logger(__name__)\n\n\ndef _packed_boundaries(\n    cu_seqlens: torch.Tensor,\n    cu_seqlens_host: Sequence[int] | None,\n    total_tokens: int,\n    name: str,\n) -> tuple[int, ...]:\n    if cu_seqlens is None:\n        raise ValueError(f\"{name} is required for NPU packed attention\")\n    if cu_seqlens.ndim != 1 or cu_seqlens.dtype not in (\n        torch.int32,\n        torch.int64,\n    ):\n        raise ValueError(f\"{name} must be a 1D int32 or int64 tensor\")\n    if cu_seqlens_host is not None and len(cu_seqlens_host) != cu_seqlens.numel():\n        raise ValueError(f\"{name} and its host copy must have the same length\")\n\n    boundaries = tuple(\n        int(value)\n        for value in (\n            cu_seqlens.tolist() if cu_seqlens_host is None else cu_seqlens_host\n        )\n    )\n    if len(boundaries) < 2 or boundaries[0] != 0:\n        raise ValueError(f\"{name} must start with 0 and contain at least one sequence\")\n    if boundaries[-1] != total_tokens:\n        raise ValueError(\n            f\"{name} must end at the packed token count {total_tokens}, \"\n            f\"got {boundaries[-1]}\"\n        )\n    if any(stop < start for start, stop in zip(boundaries[:-1], boundaries[1:])):\n        raise ValueError(f\"{name} must be non-decreasing\")","sourceCodeStart":13,"sourceCodeEnd":49,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py#L13-L49","documentation":"_packed_boundaries requires cu_seqlens tensors to be 1D and of dtype int32 or int64 — the NPU fused attention kernel accepts only these. A 2D tensor, a list, or another dtype (e.g. int16, float) fails this check.","triggerScenarios":"Calling fused_infer_attention_varlen with cu_seqlens passed as a Python list (no .ndim), a 2D tensor like shape [1, B+1], or a tensor of dtype bfloat16/float32/int16. Also triggered if a host-side list is passed where the tensor was expected.","commonSituations":"Callers constructing cu_seqlens with torch.tensor(lens).cumsum(0) without .to(torch.int32); code that passes numpy arrays or nested lists; porting code from another backend that accepted different dtypes.","solutions":["Convert: cu = torch.as_tensor(cu, dtype=torch.int32, device=q.device) and ensure it is 1D (squeeze a leading size-1 dim if present)","Build from lengths: cu = torch.nn.functional.pad(torch.tensor(lens).cumsum(0), (1,1)) then cast","Add an assertion before the call: assert cu.ndim == 1 and cu.dtype in (torch.int32, torch.int64)"],"exampleFix":"# before\ncu_seqlens_q = [0, 5, 12]  # python list -> ndim fails\n# after\ncu_seqlens_q = torch.tensor([0, 5, 12], dtype=torch.int32, device=q.device)","handlingStrategy":"type-guard","validationCode":"def to_cu_seqlens(x, device) -> torch.Tensor:\n    t = torch.as_tensor(x, dtype=torch.int32, device=device)\n    assert t.ndim == 1\n    return t","typeGuard":"def is_valid_cu_seqlens(t) -> bool:\n    return isinstance(t, torch.Tensor) and t.ndim == 1 and t.dtype in (torch.int32, torch.int64)","tryCatchPattern":null,"preventionTips":["Always construct cu_seqlens with an explicit dtype=torch.int32","Never pass raw Python lists or numpy arrays"],"tags":["npu","ascend","varlen","dtype-validation","tensor-shape"],"backgroundTag":"invalid-tensor-dtype","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}