{"record":{"id":"715a7e956d1a0f52","repo":"sgl-project/sglang","slug":"kv-canary-launch-canary-plan-kernels-torch-refere","errorCode":null,"errorMessage":"kv-canary: launch_canary_plan_kernels_torch_reference verify_capacity={verify_capacity} does not match verify_plan_out.verify_slot_indices.shape[0]={plan_verify_capacity}","messagePattern":"kv-canary: launch_canary_plan_kernels_torch_reference verify_capacity=(.+?) does not match verify_plan_out\\.verify_slot_indices\\.shape\\[0\\]=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/plan_ref.py","lineNumber":33,"sourceCode":"    write_plan_out: WritePlan,\n    req_pool_indices: torch.Tensor,\n    prefix_lens: torch.Tensor,\n    extend_seq_lens: torch.Tensor,\n    req_to_token: torch.Tensor,\n    swa_window_size: int,\n    full_to_swa_index_mapping: Optional[torch.Tensor],\n    verify_capacity: int,\n    req_to_verify_expected_tokens: Optional[torch.Tensor],\n    req_to_verify_expected_tokens_valid_lens: Optional[torch.Tensor],\n    kv_token_id_vs_position_offset: int,\n) -> None:\n    \"\"\"Python reference for :func:`launch_canary_plan_kernels`. Same signature & byte-equal semantics.\"\"\"\n    bs = int(req_pool_indices.shape[0])\n    work_device = torch.device(\"cpu\")\n\n    plan_verify_capacity = int(verify_plan_out.verify_slot_indices.shape[0])\n    if verify_capacity != plan_verify_capacity:\n        raise ValueError(\n            f\"kv-canary: launch_canary_plan_kernels_torch_reference verify_capacity={verify_capacity} does not \"\n            f\"match verify_plan_out.verify_slot_indices.shape[0]={plan_verify_capacity}\"\n        )\n    write_req_capacity = int(write_plan_out.write_seed_slot_indices.shape[0])\n\n    req_pool_indices_host = req_pool_indices.detach().to(\n        device=work_device, dtype=torch.int64\n    )\n    prefix_lens_host = prefix_lens.detach().to(device=work_device, dtype=torch.int64)\n    extend_seq_lens_host = extend_seq_lens.detach().to(\n        device=work_device, dtype=torch.int64\n    )\n    req_to_token_host = req_to_token.detach().to(device=work_device, dtype=torch.int64)\n\n    lut: Optional[torch.Tensor] = None\n    if full_to_swa_index_mapping is not None:\n        lut = full_to_swa_index_mapping.detach().to(device=work_device)\n","sourceCodeStart":15,"sourceCodeEnd":51,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/plan_ref.py#L15-L51","documentation":"The Python reference implementation of launch_canary_plan_kernels requires that the verify_capacity argument exactly equals verify_plan_out.verify_slot_indices.shape[0]. The reference iterates over the plan output slots, so a mismatched capacity would silently compute wrong results; it raises instead. This mirrors a validation the real Triton path performs.","triggerScenarios":"Calling launch_canary_plan_kernels_torch_reference with verify_capacity taken from config/topk math while verify_plan_out was allocated with a different capacity (e.g. capacity=verify_topk but the output buffer sized to topk, or a stale reused output buffer).","commonSituations":"Reusing a cached verify_plan_out across configs after changing speculative-decoding topk/capacity; computing capacity as bs*topk while the output was allocated with only bs slots.","solutions":["Allocate/resize verify_plan_out.verify_slot_indices with shape[0] == verify_capacity before the call","Recompute verify_capacity from the same expression used when the plan output was created (e.g. bs * verify_topk)","If reusing output buffers, key them on (bs, verify_capacity) so stale sizes are never passed"],"exampleFix":"# before\nverify_plan_out = alloc_verify_plan(bs * topk)\nlaunch_ref(..., verify_capacity=bs * (topk + 1), ...)\n# after\nverify_capacity = bs * (topk + 1)\nverify_plan_out = alloc_verify_plan(verify_capacity)\nlaunch_ref(..., verify_capacity=verify_capacity, ...)","handlingStrategy":"validation","validationCode":"assert verify_capacity == verify_plan_out.verify_slot_indices.shape[0], (\n    verify_capacity, verify_plan_out.verify_slot_indices.shape[0])","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive verify_capacity and the output allocation from one shared expression","Do not reuse plan output buffers across config changes (topk/capacity)"],"tags":["kv-cache","shape-mismatch","validation","speculative-decoding"],"backgroundTag":"capacity-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}