{"record":{"id":"09c4fbdbc615cfa6","repo":"sgl-project/sglang","slug":"kv-canary-scatter-req-token-ids-offsets-must-be-i","errorCode":null,"errorMessage":"kv-canary: scatter_req_token_ids offsets must be int64, got {offsets.dtype}","messagePattern":"kv-canary: scatter_req_token_ids offsets must be int64, got (.+?)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py","lineNumber":71,"sourceCode":"            f\"{tuple(offsets.shape)}\"\n        )\n    if req_pool_indices.dim() != 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be 1-D, got shape \"\n            f\"{tuple(req_pool_indices.shape)}\"\n        )\n    if pool_out.dim() != 2:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be 2-D, got shape \"\n            f\"{tuple(pool_out.shape)}\"\n        )\n    if flat_in.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids flat_in must be int64, got \"\n            f\"{flat_in.dtype}\"\n        )\n    if offsets.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids offsets must be int64, got \"\n            f\"{offsets.dtype}\"\n        )\n    if req_pool_indices.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be int64, got \"\n            f\"{req_pool_indices.dtype}\"\n        )\n    if pool_out.dtype != torch.int32:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be int32, got \"\n            f\"{pool_out.dtype}\"\n        )\n\n    bs = int(req_pool_indices.shape[0])\n    if int(offsets.shape[0]) != bs + 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids offsets length {offsets.shape[0]} != \"","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py#L53-L89","documentation":"The scatter launcher requires the CSR offsets vector to be torch.int64; offsets delimit token spans and are compared/indexed as 64-bit values in the Triton kernel. A different dtype raises TypeError before launch.","triggerScenarios":"Building offsets with torch.int32 or torch.int16 cumsum results and passing them to launch_scatter_req_token_ids_kernel.","commonSituations":"Memory-optimized ragged metadata stored as int32 in the scheduler, fed directly to the kernel.","solutions":["Cast offsets to int64: offsets.to(torch.int64)","Allocate the offsets buffer with dtype=torch.int64 from the start"],"exampleFix":"# before\noffsets = torch.cumsum(lens, 0).to(torch.int32)\n# after\noffsets = torch.cumsum(lens.to(torch.int64), 0)","handlingStrategy":"type-guard","validationCode":"assert offsets.dtype == torch.int64, offsets.dtype","typeGuard":"def is_int64(t: torch.Tensor) -> bool:\n    return t.dtype == torch.int64","tryCatchPattern":null,"preventionTips":["Build offsets in int64 from the start; converting int32 cumsums later invites overflow too"],"tags":["kv-cache","dtype-validation","torch"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}