{"record":{"id":"77c238349aca21f2","repo":"sgl-project/sglang","slug":"kv-canary-scatter-req-token-ids-pool-out-must-be-77c238","errorCode":null,"errorMessage":"kv-canary: scatter_req_token_ids pool_out must be int32, got {pool_out.dtype}","messagePattern":"kv-canary: scatter_req_token_ids pool_out must be int32, got (.+?)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py","lineNumber":81,"sourceCode":"            f\"{tuple(pool_out.shape)}\"\n        )\n    if flat_in.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids flat_in must be int64, got \"\n            f\"{flat_in.dtype}\"\n        )\n    if offsets.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids offsets must be int64, got \"\n            f\"{offsets.dtype}\"\n        )\n    if req_pool_indices.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be int64, got \"\n            f\"{req_pool_indices.dtype}\"\n        )\n    if pool_out.dtype != torch.int32:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be int32, got \"\n            f\"{pool_out.dtype}\"\n        )\n\n    bs = int(req_pool_indices.shape[0])\n    if int(offsets.shape[0]) != bs + 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids offsets length {offsets.shape[0]} != \"\n            f\"bs+1 ({bs + 1})\"\n        )\n    if bs + 1 > _SCATTER_BATCH_BLOCK:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids bs+1={bs + 1} exceeds BATCH_BLOCK=\"\n            f\"{_SCATTER_BATCH_BLOCK}; bump _SCATTER_BATCH_BLOCK if real workloads need this\"\n        )\n\n    num_tokens = int(flat_in.shape[0])\n    if num_tokens == 0:","sourceCodeStart":63,"sourceCodeEnd":99,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py#L63-L99","documentation":"Unlike the int64 inputs, pool_out must be torch.int32: the destination token-id pool is stored in 32-bit to halve memory, and the Triton kernel writes int32 words. A different dtype raises TypeError.","triggerScenarios":"Allocating pool_out with dtype=torch.int64 (or defaulting from an int64 producer) and passing it to launch_scatter_req_token_ids_kernel.","commonSituations":"Copy-pasting the allocation of the int64 inputs for the output pool; unifying dtypes across a pipeline and forgetting pool_out is the exception.","solutions":["Allocate pool_out with dtype=torch.int32","Cast an existing buffer: pool_out.to(torch.int32) (prefer allocating correctly to avoid a copy)"],"exampleFix":"# before\npool_out = torch.empty((num_reqs, max_len), dtype=torch.int64)\n# after\npool_out = torch.empty((num_reqs, max_len), dtype=torch.int32)","handlingStrategy":"type-guard","validationCode":"assert pool_out.dtype == torch.int32, pool_out.dtype","typeGuard":"def is_int32(t: torch.Tensor) -> bool:\n    return t.dtype == torch.int32","tryCatchPattern":null,"preventionTips":["Remember the asymmetry: inputs int64, pool_out int32"],"tags":["kv-cache","dtype-validation","torch"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}