{"record":{"id":"969728e3113b7acd","repo":"sgl-project/sglang","slug":"kv-canary-offsets-kernel-bs-must-be-in-0-plan","errorCode":null,"errorMessage":"kv-canary: offsets kernel bs must be in [0, {_PLAN_BS_BLOCK_SIZE}], got {bs}","messagePattern":"kv-canary: offsets kernel bs must be in \\[0, (.+?)\\], got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/plan/offsets_kernel.py","lineNumber":153,"sourceCode":"\n    _require_dtype(req_pool_indices, \"req_pool_indices\", torch.int64)\n    _require_dtype(prefix_lens, \"prefix_lens\", torch.int64)\n    _require_dtype(extend_seq_lens, \"extend_seq_lens\", torch.int64)\n    _require_dtype(req_to_token, \"req_to_token\", torch.int32)\n    _require_dtype(lut_tensor, \"lut_tensor\", torch.int64)\n    _require_dtype(\n        out_verify_offsets_scratch, \"out_verify_offsets_scratch\", torch.int64\n    )\n    _require_dtype(out_write_offsets, \"out_write_offsets\", torch.int64)\n    _require_dtype(\n        out_write_seed_slot_indices, \"out_write_seed_slot_indices\", torch.int64\n    )\n    _require_dtype(out_verify_num_valid, \"out_verify_num_valid\", torch.int32)\n    _require_dtype(out_verify_enable, \"out_verify_enable\", torch.int32)\n    _require_dtype(out_write_num_valid_reqs, \"out_write_num_valid_reqs\", torch.int32)\n\n    if bs < 0 or bs > _PLAN_BS_BLOCK_SIZE:\n        raise ValueError(\n            f\"kv-canary: offsets kernel bs must be in [0, {_PLAN_BS_BLOCK_SIZE}], got {bs}\"\n        )\n    if write_offsets_len <= 0:\n        raise ValueError(\n            f\"kv-canary: write_offsets_len must be positive, got {write_offsets_len}\"\n        )\n    if write_req_capacity < 0:\n        raise ValueError(\n            f\"kv-canary: write_req_capacity must be non-negative, got {write_req_capacity}\"\n        )\n    if verify_capacity < 0:\n        raise ValueError(\n            f\"kv-canary: verify_capacity must be non-negative, got {verify_capacity}\"\n        )\n    if req_to_token_stride0 <= 0:\n        raise ValueError(\n            f\"kv-canary: req_to_token_stride0 must be positive, got {req_to_token_stride0}\"\n        )","sourceCodeStart":135,"sourceCodeEnd":171,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/plan/offsets_kernel.py#L135-L171","documentation":"The offsets kernel is compiled for a fixed batch-size block, so the runtime batch size must be within [0, _PLAN_BS_BLOCK_SIZE]. Values outside that range would break kernel indexing assumptions.","triggerScenarios":"Calling launch_plan_offsets_kernel with bs < 0 or bs > _PLAN_BS_BLOCK_SIZE (e.g. a running-batch larger than the compiled block, typically 256).","commonSituations":"High-concurrency serving where the running batch exceeds the kernel's fixed block size; negative bs from a buggy upstream trim/slice.","solutions":["Split the launch into chunks of at most _PLAN_BS_BLOCK_SIZE requests","Bump _PLAN_BS_BLOCK_SIZE in the kernel module if real workloads need larger blocks (as the sibling error message suggests)","Check upstream slicing logic if bs is negative"],"exampleFix":"// before\nlaunch_plan_offsets_kernel(bs=len(reqs), ...)\n// after\nfor i in range(0, len(reqs), _PLAN_BS_BLOCK_SIZE):\n    launch_plan_offsets_kernel(bs=min(_PLAN_BS_BLOCK_SIZE, len(reqs)-i), ...)","handlingStrategy":"validation","validationCode":"from python.sglang.kernels.ops.kv_canary.plan.offsets_kernel import _PLAN_BS_BLOCK_SIZE\nassert 0 <= bs <= _PLAN_BS_BLOCK_SIZE, f\"bs={bs} exceeds kernel block {_PLAN_BS_BLOCK_SIZE}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Chunk plan launches at _PLAN_BS_BLOCK_SIZE","Monitor running-batch size against the kernel block size in CI load tests"],"tags":["kv-canary","batch-size","bounds-check"],"backgroundTag":"batch-size-exceeds-limit","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}