{"record":{"id":"97672318970261b3","repo":"sgl-project/sglang","slug":"kv-canary-scatter-req-token-ids-offsets-length-o","errorCode":null,"errorMessage":"kv-canary: scatter_req_token_ids offsets length {offsets.shape[0]} != bs+1 ({bs + 1})","messagePattern":"kv-canary: scatter_req_token_ids offsets length (.+?) != bs\\+1 \\((.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py","lineNumber":88,"sourceCode":"    if offsets.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids offsets must be int64, got \"\n            f\"{offsets.dtype}\"\n        )\n    if req_pool_indices.dtype != torch.int64:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids req_pool_indices must be int64, got \"\n            f\"{req_pool_indices.dtype}\"\n        )\n    if pool_out.dtype != torch.int32:\n        raise TypeError(\n            f\"kv-canary: scatter_req_token_ids pool_out must be int32, got \"\n            f\"{pool_out.dtype}\"\n        )\n\n    bs = int(req_pool_indices.shape[0])\n    if int(offsets.shape[0]) != bs + 1:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids offsets length {offsets.shape[0]} != \"\n            f\"bs+1 ({bs + 1})\"\n        )\n    if bs + 1 > _SCATTER_BATCH_BLOCK:\n        raise ValueError(\n            f\"kv-canary: scatter_req_token_ids bs+1={bs + 1} exceeds BATCH_BLOCK=\"\n            f\"{_SCATTER_BATCH_BLOCK}; bump _SCATTER_BATCH_BLOCK if real workloads need this\"\n        )\n\n    num_tokens = int(flat_in.shape[0])\n    if num_tokens == 0:\n        return\n\n    pool_stride0 = int(pool_out.stride(0))\n    pool_max_context_len = int(pool_out.shape[1])\n\n    grid = (triton.cdiv(num_tokens, _SCATTER_TOKEN_BLOCK),)\n    _scatter_req_token_ids_kernel[grid](","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/scatter_req_token_ids.py#L70-L106","documentation":"offsets must be a CSR prefix vector of exactly length bs+1 (where bs = req_pool_indices.shape[0]): offsets[i]:offsets[i+1] is request i's span in flat_in. Any other length raises ValueError because the kernel reads bs+1 offsets per block.","triggerScenarios":"Passing bs offsets (per-request lengths instead of prefix sums), or bs+2, or an offsets vector built for a different batch size than req_pool_indices.","commonSituations":"Off-by-one confusion between lengths and prefix offsets; stale offsets tensor from a previous batch after the batch size changed.","solutions":["Build offsets as a length-bs+1 prefix sum: offsets = torch.zeros(bs+1, dtype=torch.int64); offsets[1:] = lens.cumsum(0)","Rebuild offsets whenever the batch composition changes; do not cache across scheduler iterations","Assert offsets.shape[0] == req_pool_indices.shape[0] + 1 before calling"],"exampleFix":"# before\noffsets = lens  # lengths, not prefix offsets\n# after\noffsets = torch.zeros(lens.shape[0] + 1, dtype=torch.int64)\noffsets[1:] = torch.cumsum(lens, 0)","handlingStrategy":"validation","validationCode":"assert offsets.shape[0] == req_pool_indices.shape[0] + 1, (offsets.shape, req_pool_indices.shape)\nassert (torch.diff(offsets) >= 0).all()","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never pass lengths where prefix offsets are expected; rebuild offsets each batch"],"tags":["kv-cache","shape-validation","csr-offsets","off-by-one"],"backgroundTag":"tensor-shape-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}