{"record":{"id":"fad7531a3148581a","repo":"sgl-project/sglang","slug":"kv-canary-scatter-req-token-ids-req-pool-indices-fad753","errorCode":null,"errorMessage":"kv-canary: scatter_req_token_ids req_pool_indices must be int64, got {req_pool_indices.dtype}","messagePattern":"kv-canary: scatter_req_token_ids req_pool_indices must be int64, got (.+?)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py","lineNumber":76,"sourceCode":"            f\"{tuple(req_pool_indices.shape)}\"\n        )\n    if pool_out.dim() != 2:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be 2-D, got shape \"\n            f\"{tuple(pool_out.shape)}\"\n        )\n    if flat_in.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids flat_in must be int64, got \"\n            f\"{flat_in.dtype}\"\n        )\n    if offsets.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids offsets must be int64, got \"\n            f\"{offsets.dtype}\"\n        )\n    if req_pool_indices.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be int64, got \"\n            f\"{req_pool_indices.dtype}\"\n        )\n    if pool_out.dtype != torch.int32:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be int32, got \"\n            f\"{pool_out.dtype}\"\n        )\n\n    bs = int(req_pool_indices.shape[0])\n    if int(offsets.shape[0]) != bs + 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids offsets length {offsets.shape[0]} != \"\n            f\"bs+1 ({bs + 1})\"\n        )\n    if bs + 1 > _SCATTER_BATCH_BLOCK:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids bs+1={bs + 1} exceeds BATCH_BLOCK=\"","sourceCodeStart":58,"sourceCodeEnd":94,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py#L58-L94","documentation":"The scatter launcher requires req_pool_indices to be torch.int64, matching the request-pool index ABI used across kv-canary kernels. Any other dtype raises TypeError.","triggerScenarios":"Passing int32 request-pool indices (common when the allocator pool is int32) to launch_scatter_req_token_ids_kernel.","commonSituations":"Scheduler request-pool tensors stored as int32 for compactness; converting from numpy arrays that default to int32 on Windows.","solutions":["Cast: req_pool_indices.to(torch.int64)","Keep one canonical int64 copy of req_pool_indices for kv-canary calls"],"exampleFix":"# before\nlaunch_scatter(..., req_pool_indices=rp_int32)\n# after\nlaunch_scatter(..., req_pool_indices=rp_int32.to(torch.int64))","handlingStrategy":"type-guard","validationCode":"assert req_pool_indices.dtype == torch.int64, req_pool_indices.dtype","typeGuard":"def is_int64(t: torch.Tensor) -> bool:\n    return t.dtype == torch.int64","tryCatchPattern":null,"preventionTips":["Cache an int64 view/copy of the request pool indices for kv-canary calls"],"tags":["kv-cache","dtype-validation","torch"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}