{"record":{"id":"b1841d813ee096bc","repo":"sgl-project/sglang","slug":"kv-canary-scatter-req-token-ids-pool-out-must-be","errorCode":null,"errorMessage":"kv-canary: scatter_req_token_ids pool_out must be 2-D, got shape {tuple(pool_out.shape)}","messagePattern":"kv-canary: scatter_req_token_ids pool_out must be 2-D, got shape (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py","lineNumber":61,"sourceCode":"          registers for the workloads kv-canary handles (``bs <= a few thousand``).\n    \"\"\"\n    if flat_in.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids flat_in must be 1-D, got shape \"\n            f\"{tuple(flat_in.shape)}\"\n        )\n    if offsets.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids offsets must be 1-D, got shape \"\n            f\"{tuple(offsets.shape)}\"\n        )\n    if req_pool_indices.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be 1-D, got shape \"\n            f\"{tuple(req_pool_indices.shape)}\"\n        )\n    if pool_out.dim() != 2:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be 2-D, got shape \"\n            f\"{tuple(pool_out.shape)}\"\n        )\n    if flat_in.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids flat_in must be int64, got \"\n            f\"{flat_in.dtype}\"\n        )\n    if offsets.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids offsets must be int64, got \"\n            f\"{offsets.dtype}\"\n        )\n    if req_pool_indices.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be int64, got \"\n            f\"{req_pool_indices.dtype}\"\n        )","sourceCodeStart":43,"sourceCodeEnd":79,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py#L43-L79","documentation":"The scatter launcher requires pool_out — the [num_reqs, max_context_len] destination pool being filled with token ids — to be exactly 2-D. Any other rank raises ValueError since the kernel indexes pool_out[req, pos].","triggerScenarios":"Passing a 1-D flattened pool, a 3-D [layers, reqs, len] pool, or a pool view with an extra singleton dim to launch_scatter_req_token_ids_kernel.","commonSituations":"Slicing a multi-layer KV metadata buffer and forgetting to select the layer dim; passing the whole stack instead of pool[layer].","solutions":["Index down to 2-D: pool_out = token_pool[layer] or .squeeze(extra_dim)","Allocate the destination as torch.empty((num_reqs, max_context_len), dtype=torch.int32)"],"exampleFix":"# before\nlaunch_scatter(..., pool_out=token_pool_3d)\n# after\nlaunch_scatter(..., pool_out=token_pool_3d[0])  # [num_reqs, max_context_len]","handlingStrategy":"type-guard","validationCode":"assert pool_out.dim() == 2, pool_out.shape","typeGuard":"def is_2d(t: torch.Tensor) -> bool:\n    return t.dim() == 2","tryCatchPattern":null,"preventionTips":["Allocate pool_out as torch.empty((num_reqs, max_context_len), dtype=torch.int32)"],"tags":["kv-cache","shape-validation","tensor-rank"],"backgroundTag":"tensor-shape-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}