{"record":{"id":"613f3333e7239e01","repo":"sgl-project/sglang","slug":"kv-canary-scatter-req-token-ids-offsets-must-be-1","errorCode":null,"errorMessage":"kv-canary: scatter_req_token_ids offsets must be 1-D, got shape {tuple(offsets.shape)}","messagePattern":"kv-canary: scatter_req_token_ids offsets must be 1-D, got shape (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py","lineNumber":51,"sourceCode":"        flat_in: ``[total_tokens]`` int64 device tensor of objects, flattened\n            per-req in req order.\n        offsets: ``[bs + 1]`` int64 device tensor (host-computed cumsum of per-req\n            lengths). ``offsets[bs] == total_tokens``.\n        req_pool_indices: ``[bs]`` int64 device tensor of pool row indices.\n        pool_out: ``[max_reqs, max_context_len]`` int32 device tensor of objects.\n            Mutated in-place; rows not addressed by ``req_pool_indices`` are untouched.\n\n    Implementation notes:\n        - Linear scan over ``offsets`` (``BATCH_BLOCK >= bs + 1``); fits easily in\n          registers for the workloads kv-canary handles (``bs <= a few thousand``).\n    \"\"\"\n    if flat_in.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids flat_in must be 1-D, got shape \"\n            f\"{tuple(flat_in.shape)}\"\n        )\n    if offsets.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids offsets must be 1-D, got shape \"\n            f\"{tuple(offsets.shape)}\"\n        )\n    if req_pool_indices.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be 1-D, got shape \"\n            f\"{tuple(req_pool_indices.shape)}\"\n        )\n    if pool_out.dim() != 2:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be 2-D, got shape \"\n            f\"{tuple(pool_out.shape)}\"\n        )\n    if flat_in.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids flat_in must be int64, got \"\n            f\"{flat_in.dtype}\"\n        )","sourceCodeStart":33,"sourceCodeEnd":69,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py#L33-L69","documentation":"The scatter launcher requires offsets to be 1-D: it is the CSR-style prefix-offset vector of length bs+1 delimiting each request's token span in flat_in. Passing a 2-D/scalar offsets tensor raises ValueError before kernel launch.","triggerScenarios":"Passing offsets with an extra dimension (e.g. shape [1, bs+1] after a keepdim operation) or a 0-D scalar to launch_scatter_req_token_ids_kernel.","commonSituations":"Offsets produced by torch.cumsum(..., keepdim=True) or slicing a 2-D buffer; bugs in ragged-batch construction.","solutions":["Squeeze the tensor: offsets = offsets.squeeze(0) or .reshape(-1)","Build offsets with 1-D cumsum: torch.zeros(bs+1, dtype=torch.int64) then fill","Add an assertion offsets.dim() == 1 right after construction in the caller"],"exampleFix":"# before\noffsets = torch.cumsum(lens, 0, keepdim=True)  # wrong dims\n# after\noffsets = torch.zeros(bs + 1, dtype=torch.int64)\noffsets[1:] = torch.cumsum(lens, 0)","handlingStrategy":"type-guard","validationCode":"assert offsets.dim() == 1, offsets.shape","typeGuard":"def is_1d(t: torch.Tensor) -> bool:\n    return t.dim() == 1","tryCatchPattern":null,"preventionTips":["Construct offsets into a preallocated 1-D int64 buffer; avoid keepdim=True in cumsum"],"tags":["kv-cache","shape-validation","tensor-rank"],"backgroundTag":"tensor-shape-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}