{"record":{"id":"9f21eed65352a437","repo":"sgl-project/sglang","slug":"kv-canary-scatter-req-token-ids-req-pool-indices","errorCode":null,"errorMessage":"kv-canary: scatter_req_token_ids req_pool_indices must be 1-D, got shape {tuple(req_pool_indices.shape)}","messagePattern":"kv-canary: scatter_req_token_ids req_pool_indices must be 1-D, got shape (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py","lineNumber":56,"sourceCode":"        pool_out: ``[max_reqs, max_context_len]`` int32 device tensor of objects.\n            Mutated in-place; rows not addressed by ``req_pool_indices`` are untouched.\n\n    Implementation notes:\n        - Linear scan over ``offsets`` (``BATCH_BLOCK >= bs + 1``); fits easily in\n          registers for the workloads kv-canary handles (``bs <= a few thousand``).\n    \"\"\"\n    if flat_in.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids flat_in must be 1-D, got shape \"\n            f\"{tuple(flat_in.shape)}\"\n        )\n    if offsets.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids offsets must be 1-D, got shape \"\n            f\"{tuple(offsets.shape)}\"\n        )\n    if req_pool_indices.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be 1-D, got shape \"\n            f\"{tuple(req_pool_indices.shape)}\"\n        )\n    if pool_out.dim() != 2:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be 2-D, got shape \"\n            f\"{tuple(pool_out.shape)}\"\n        )\n    if flat_in.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids flat_in must be int64, got \"\n            f\"{flat_in.dtype}\"\n        )\n    if offsets.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids offsets must be int64, got \"\n            f\"{offsets.dtype}\"\n        )","sourceCodeStart":38,"sourceCodeEnd":74,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py#L38-L74","documentation":"The scatter launcher requires req_pool_indices (per-request request-pool row indices) to be a 1-D tensor of length bs. A higher-rank tensor raises ValueError because the kernel indexes it by batch row.","triggerScenarios":"Passing req_pool_indices of shape [bs, 1] or [1, bs], e.g. after unsqueezing for another kernel's grid, to launch_scatter_req_token_ids_kernel.","commonSituations":"Reusing tensors shaped for a different kernel ABI; batching code that adds a dummy dimension.","solutions":["Apply .squeeze(-1) or .reshape(-1) to restore 1-D","Keep a canonical 1-D int64 req_pool_indices in the batch state and only reshape at other call sites"],"exampleFix":"# before\nlaunch_scatter(..., req_pool_indices=rp.unsqueeze(-1))\n# after\nlaunch_scatter(..., req_pool_indices=rp)","handlingStrategy":"type-guard","validationCode":"assert req_pool_indices.dim() == 1, req_pool_indices.shape","typeGuard":"def is_1d(t: torch.Tensor) -> bool:\n    return t.dim() == 1","tryCatchPattern":null,"preventionTips":["Keep one canonical 1-D req_pool_indices; reshape only at other kernels' call sites"],"tags":["kv-cache","shape-validation","tensor-rank"],"backgroundTag":"tensor-shape-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}