{"record":{"id":"afe5fa4ca7a575e2","repo":"sgl-project/sglang","slug":"name-must-end-at-the-packed-token-count-total-t","errorCode":null,"errorMessage":"{name} must end at the packed token count {total_tokens}, got {boundaries[-1]}","messagePattern":"(.+?) must end at the packed token count (.+?), got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py","lineNumber":44,"sourceCode":"        raise ValueError(f\"{name} is required for NPU packed attention\")\n    if cu_seqlens.ndim != 1 or cu_seqlens.dtype not in (\n        torch.int32,\n        torch.int64,\n    ):\n        raise ValueError(f\"{name} must be a 1D int32 or int64 tensor\")\n    if cu_seqlens_host is not None and len(cu_seqlens_host) != cu_seqlens.numel():\n        raise ValueError(f\"{name} and its host copy must have the same length\")\n\n    boundaries = tuple(\n        int(value)\n        for value in (\n            cu_seqlens.tolist() if cu_seqlens_host is None else cu_seqlens_host\n        )\n    )\n    if len(boundaries) < 2 or boundaries[0] != 0:\n        raise ValueError(f\"{name} must start with 0 and contain at least one sequence\")\n    if boundaries[-1] != total_tokens:\n        raise ValueError(\n            f\"{name} must end at the packed token count {total_tokens}, \"\n            f\"got {boundaries[-1]}\"\n        )\n    if any(stop < start for start, stop in zip(boundaries[:-1], boundaries[1:])):\n        raise ValueError(f\"{name} must be non-decreasing\")\n    return boundaries\n\n\ndef fused_infer_attention_varlen(\n    q: torch.Tensor,\n    k: torch.Tensor,\n    v: torch.Tensor,\n    cu_seqlens_q: torch.Tensor,\n    cu_seqlens_k: torch.Tensor,\n    *,\n    cu_seqlens_q_host: Sequence[int] | None = None,\n    cu_seqlens_k_host: Sequence[int] | None = None,\n    softmax_scale: float | None = None,","sourceCodeStart":26,"sourceCodeEnd":62,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py#L26-L62","documentation":"_packed_boundaries verifies that the last cu_seqlens entry equals the total token count of the corresponding tensor (q.shape[0] or k.shape[0] in TND layout). A mismatch means the boundaries do not cover every token in the packed tensor — tokens would be silently dropped or over-counted.","triggerScenarios":"Passing q with T=100 but cu_seqlens_k ending at 90 (or cu_seqlens_q ending at 120); typically after truncating/masking a packed batch on one side only, or reusing cu_seqlens from a different batch.","commonSituations":"Padded packed batches where padding tokens were sliced off the tensor but not from the boundaries; batched ring-attention chunking that splits KV without adjusting the KV boundaries; stale metadata reused across iterations.","solutions":["Recompute cu_seqlens from the actual tensor: ensure boundaries[-1] == tensor.shape[0]","When slicing tokens off the end, also clip the boundary values: cu = torch.clamp(cu, max=new_T) keeping monotonicity","For ring-KV chunks, pass chunk-local cu_seqlens_k whose final entry is the chunk's token count"],"exampleFix":"# before\nq = q[:100]  # truncated\ncu_q = torch.tensor([0, 60, 130], dtype=torch.int32)  # ends at 130\n# after\nq = q[:100]\ncu_q = torch.tensor([0, 60, 100], dtype=torch.int32)  # ends at q.shape[0]","handlingStrategy":"validation","validationCode":"assert cu_q[-1].item() == q.shape[0] and cu_k[-1].item() == k.shape[0]","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Recompute cu_seqlens after any tensor truncation","In ring-KV chunking, keep boundaries chunk-local"],"tags":["npu","ascend","varlen","packed-sequences","shape-validation"],"backgroundTag":"malformed-cumulative-seqlens","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}