{"record":{"id":"66f3cfd2e3556352","repo":"sgl-project/sglang","slug":"kv-canary-name-length-must-be-expected-got","errorCode":null,"errorMessage":"kv-canary: {name} length must be {expected}, got {actual}","messagePattern":"kv-canary: (.+?) length must be (.+?), got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/plan/utils.py","lineNumber":50,"sourceCode":"def _require_1d(tensor: torch.Tensor, name: str) -> None:\n    if tensor.ndim != 1:\n        raise ValueError(\n            f\"kv-canary: {name} must be 1-D, got shape {tuple(tensor.shape)}\"\n        )\n\n\ndef _require_2d(tensor: torch.Tensor, name: str) -> None:\n    if tensor.ndim != 2:\n        raise ValueError(\n            f\"kv-canary: {name} must be 2-D, got shape {tuple(tensor.shape)}\"\n        )\n\n\ndef _require_len(tensor: torch.Tensor, name: str, expected: int) -> None:\n    _require_1d(tensor=tensor, name=name)\n    actual = int(tensor.shape[0])\n    if actual != expected:\n        raise ValueError(f\"kv-canary: {name} length must be {expected}, got {actual}\")\n\n\ndef _require_min_len(tensor: torch.Tensor, name: str, minimum: int) -> None:\n    _require_1d(tensor=tensor, name=name)\n    actual = int(tensor.shape[0])\n    if actual < minimum:\n        raise ValueError(f\"kv-canary: {name} length must be >= {minimum}, got {actual}\")\n\n\ndef _require_same_device(\n    reference: torch.Tensor,\n    reference_name: str,\n    tensors: tuple[tuple[torch.Tensor, str], ...],\n) -> None:\n    for tensor, name in tensors:\n        if tensor.device != reference.device:\n            raise ValueError(\n                f\"kv-canary: {name} must be on {reference_name}'s device \"","sourceCodeStart":32,"sourceCodeEnd":68,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/plan/utils.py#L32-L68","documentation":"Per-request input vectors (req_pool_indices, prefix_lens, extend_seq_lens, etc.) must have length exactly equal to the declared batch size bs; the kernel reads one element per request with no bounds slack.","triggerScenarios":"Calling launch_plan_offsets_kernel where a per-request input's shape[0] != bs — e.g. tensors sliced to a subset of the batch, or bs computed from a different list than the tensors.","commonSituations":"Batch filtering (removing finished requests) applied to bs but not the tensors, or vice versa; off-by-one in slicing during chunked prefill.","solutions":["Recompute all per-request tensors and bs from the same request list so lengths agree by construction","Print each tensor's shape[0] vs bs to find the diverging input named in the message"],"exampleFix":"// before\nbs = len(all_reqs)\nlens = lens_for_active_only  # shorter than all_reqs\n// after\nactive = [r for r in all_reqs if not r.finished]\nbs = len(active)\nlens = build_lens(active)","handlingStrategy":"validation","validationCode":"assert all(t.shape[0] == bs for t in (req_pool_indices, prefix_lens, extend_seq_lens))","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Build bs and all per-request tensors from the same request list in one place"],"tags":["kv-canary","length-mismatch","validation"],"backgroundTag":"tensor-length-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}