{"record":{"id":"da335ec716fa139a","repo":"sgl-project/sglang","slug":"name-and-its-host-copy-must-have-the-same-length","errorCode":null,"errorMessage":"{name} and its host copy must have the same length","messagePattern":"(.+?) and its host copy must have the same length","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py","lineNumber":33,"sourceCode":"\nlogger = init_logger(__name__)\n\n\ndef _packed_boundaries(\n    cu_seqlens: torch.Tensor,\n    cu_seqlens_host: Sequence[int] | None,\n    total_tokens: int,\n    name: str,\n) -> tuple[int, ...]:\n    if cu_seqlens is None:\n        raise ValueError(f\"{name} is required for NPU packed attention\")\n    if cu_seqlens.ndim != 1 or cu_seqlens.dtype not in (\n        torch.int32,\n        torch.int64,\n    ):\n        raise ValueError(f\"{name} must be a 1D int32 or int64 tensor\")\n    if cu_seqlens_host is not None and len(cu_seqlens_host) != cu_seqlens.numel():\n        raise ValueError(f\"{name} and its host copy must have the same length\")\n\n    boundaries = tuple(\n        int(value)\n        for value in (\n            cu_seqlens.tolist() if cu_seqlens_host is None else cu_seqlens_host\n        )\n    )\n    if len(boundaries) < 2 or boundaries[0] != 0:\n        raise ValueError(f\"{name} must start with 0 and contain at least one sequence\")\n    if boundaries[-1] != total_tokens:\n        raise ValueError(\n            f\"{name} must end at the packed token count {total_tokens}, \"\n            f\"got {boundaries[-1]}\"\n        )\n    if any(stop < start for start, stop in zip(boundaries[:-1], boundaries[1:])):\n        raise ValueError(f\"{name} must be non-decreasing\")\n    return boundaries\n","sourceCodeStart":15,"sourceCodeEnd":51,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/attention/backends/ascend_fa.py#L15-L51","documentation":"When a host-side copy of cu_seqlens is supplied (cu_seqlens_host) to avoid a device-to-host sync, its length must equal cu_seqlens.numel(). A mismatch means the device tensor and host list describe different numbers of sequences and the kernel would compute wrong boundaries.","triggerScenarios":"Calling fused_infer_attention_varlen with cu_seqlens_q_host (or _k_host) whose len() differs from the corresponding device tensor's element count — e.g. host list from a previous batch, or off-by-one (len B vs B+1 boundaries).","commonSituations":"Caching the host copy across batches and forgetting to update it; building the host copy from per-batch lengths but the device tensor from a different (e.g. filtered) batch; off-by-one boundary conventions.","solutions":["Regenerate the host copy from the same source as the device tensor in the same step: host = cu.tolist() before any batch mutation","Ensure both use the inclusive convention: B+1 entries for B sequences","Add an assert len(cu_host) == cu.numel() right before the call"],"exampleFix":"# before\ncu_q = torch.tensor([0,5,12], dtype=torch.int32)\ncu_q_host = [0, 5, 12, 20]  # stale, longer\n# after\ncu_q_host = cu_q.tolist()\nassert len(cu_q_host) == cu_q.numel()","handlingStrategy":"validation","validationCode":"if cu_host is not None:\n    assert len(cu_host) == cu.numel(), \"host copy out of sync with device tensor\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive the host copy from the device tensor (cu.tolist()) in the same frame","Never cache host copies across batches"],"tags":["npu","ascend","varlen","host-device-sync","validation"],"backgroundTag":"tensor-length-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}