{"record":{"id":"c0a0570391ff09ac","repo":"sgl-project/sglang","slug":"kv-canary-scatter-req-token-ids-flat-in-must-be-i","errorCode":null,"errorMessage":"kv-canary: scatter_req_token_ids flat_in must be int64, got {flat_in.dtype}","messagePattern":"kv-canary: scatter_req_token_ids flat_in must be int64, got (.+?)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py","lineNumber":66,"sourceCode":"            f\"{tuple(flat_in.shape)}\"\n        )\n    if offsets.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids offsets must be 1-D, got shape \"\n            f\"{tuple(offsets.shape)}\"\n        )\n    if req_pool_indices.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be 1-D, got shape \"\n            f\"{tuple(req_pool_indices.shape)}\"\n        )\n    if pool_out.dim() != 2:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be 2-D, got shape \"\n            f\"{tuple(pool_out.shape)}\"\n        )\n    if flat_in.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids flat_in must be int64, got \"\n            f\"{flat_in.dtype}\"\n        )\n    if offsets.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids offsets must be int64, got \"\n            f\"{offsets.dtype}\"\n        )\n    if req_pool_indices.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be int64, got \"\n            f\"{req_pool_indices.dtype}\"\n        )\n    if pool_out.dtype != torch.int32:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be int32, got \"\n            f\"{pool_out.dtype}\"\n        )","sourceCodeStart":48,"sourceCodeEnd":84,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py#L48-L84","documentation":"The scatter kernel reads raw int64 words from flat_in (token ids with sentinel semantics encoded in 64-bit), so the launcher enforces dtype torch.int64 and raises TypeError otherwise. int32 inputs would be misread as pairs, corrupting the scatter.","triggerScenarios":"Calling launch_scatter_req_token_ids_kernel with flat_in of dtype torch.int32, torch.uint8, etc.","commonSituations":"Downstream code that stores token ids as int32 for memory efficiency; tensors coming from a tokenizer configured with a narrower dtype.","solutions":["Cast: flat_in = flat_in.to(torch.int64) at the call site","Ensure the producer of flat_in allocates with dtype=torch.int64"],"exampleFix":"# before\nlaunch_scatter(..., flat_in=flat_in_int32)\n# after\nlaunch_scatter(..., flat_in=flat_in_int32.to(torch.int64))","handlingStrategy":"type-guard","validationCode":"assert flat_in.dtype == torch.int64, flat_in.dtype","typeGuard":"def is_int64(t: torch.Tensor) -> bool:\n    return t.dtype == torch.int64","tryCatchPattern":null,"preventionTips":["Cast at the boundary: flat_in.to(torch.int64, copy=False)"],"tags":["kv-cache","dtype-validation","torch"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}