{"record":{"id":"bf3b7d057ca152b0","repo":"sgl-project/sglang","slug":"name-is-required-for-npu-packed-attention","errorCode":null,"errorMessage":"{name} is required for NPU packed attention","messagePattern":"(.+?) is required for NPU packed attention","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py","lineNumber":26,"sourceCode":"    AttentionBackend,\n    AttentionImpl,\n    AttentionMetadata,\n    AttentionMetadataBuilder,\n)\nfrom sglang.multimodal_gen.runtime.platforms import AttentionBackendEnum\nfrom sglang.multimodal_gen.runtime.utils.logging_utils import init_logger\n\nlogger = init_logger(__name__)\n\n\ndef _packed_boundaries(\n    cu_seqlens: torch.Tensor,\n    cu_seqlens_host: Sequence[int] | None,\n    total_tokens: int,\n    name: str,\n) -> tuple[int, ...]:\n    if cu_seqlens is None:\n        raise ValueError(f\"{name} is required for NPU packed attention\")\n    if cu_seqlens.ndim != 1 or cu_seqlens.dtype not in (\n        torch.int32,\n        torch.int64,\n    ):\n        raise ValueError(f\"{name} must be a 1D int32 or int64 tensor\")\n    if cu_seqlens_host is not None and len(cu_seqlens_host) != cu_seqlens.numel():\n        raise ValueError(f\"{name} and its host copy must have the same length\")\n\n    boundaries = tuple(\n        int(value)\n        for value in (\n            cu_seqlens.tolist() if cu_seqlens_host is None else cu_seqlens_host\n        )\n    )\n    if len(boundaries) < 2 or boundaries[0] != 0:\n        raise ValueError(f\"{name} must start with 0 and contain at least one sequence\")\n    if boundaries[-1] != total_tokens:\n        raise ValueError(","sourceCodeStart":8,"sourceCodeEnd":44,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py#L8-L44","documentation":"_packed_boundaries validates the cumulative-seqlens tensors required by NPU (Ascend) packed/varlen attention. cu_seqlens_q (and cu_seqlens_k) must not be None; they encode per-sequence token boundaries as a 1D tensor starting at 0. Passing None means the caller skipped required inputs.","triggerScenarios":"Calling fused_infer_attention_varlen (directly or via forward_varlen/forward_ring_kv_chunk) with cu_seqlens_q or cu_seqlens_k set to None; name is substituted with 'cu_seqlens_q' or 'cu_seqlens_k'.","commonSituations":"A varlen forward path that only materializes cu_seqlens in some branches (e.g. batch-of-one shortcut skips building them); refactoring that passes the host copy but not the device tensor, or vice versa.","solutions":["Always build and pass cu_seqlens_q and cu_seqlens_k as int32/int64 tensors: [0, len0, len0+len1, ...]","Audit call sites in forward_varlen and forward_ring_kv_chunk to ensure neither argument is dropped","If sequences are uniform you still need cu_seqlens (repeat of cumsum), not None"],"exampleFix":"# before\nout = fused_infer_attention_varlen(q, k, v, cu_seqlens_q=None, cu_seqlens_k=None)\n# after\ncu_q = torch.tensor([0, 5, 12, 20], dtype=torch.int32, device=q.device)\ncu_k = torch.tensor([0, 9, 21, 34], dtype=torch.int32, device=k.device)\nout = fused_infer_attention_varlen(q, k, v, cu_seqlens_q=cu_q, cu_seqlens_k=cu_k)","handlingStrategy":"validation","validationCode":"assert cu_seqlens_q is not None and cu_seqlens_k is not None, \"cu_seqlens required for varlen NPU attention\"","typeGuard":"def has_cu_seqlens(*args: torch.Tensor | None) -> bool:\n    return all(t is not None for t in args)","tryCatchPattern":null,"preventionTips":["Make cu_seqlens mandatory parameters (not Optional) in your wrapper API"],"tags":["npu","ascend","varlen","attention","packed-sequences"],"backgroundTag":"missing-cumulative-seqlens","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}