{"record":{"id":"62efcc2684393c8a","repo":"sgl-project/sglang","slug":"hisparse-speculative-swap-requires-2-4-steps-got","errorCode":null,"errorMessage":"HiSparse speculative swap requires 2-4 steps, got {num_steps}.","messagePattern":"HiSparse speculative swap requires 2-4 steps, got (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kvcache/hisparse.py","lineNumber":96,"sourceCode":"    device_buffer: torch.Tensor,\n    top_k_device_locs: torch.Tensor,\n    req_pool_indices: torch.Tensor,\n    seq_lens: torch.Tensor,\n    state: HiSparseSpecState,\n    num_real_reqs: torch.Tensor,\n    miss_src: torch.Tensor | None = None,\n    miss_dst: torch.Tensor | None = None,\n    miss_count: torch.Tensor | None = None,\n) -> None:\n    \"\"\"Resolve all speculative steps and swap unique misses in one launch pair.\n\n    Optional miss-plan outputs use the same protocol as the single-step HiSparse\n    kernel, so shared-index layers can replay only the Host-to-GPU copies with\n    ``copy_cache_planned_mla``.\n    \"\"\"\n    _, num_steps, num_top_k = top_k_tokens.shape\n    if not 2 <= num_steps <= 4:\n        raise ValueError(\n            f\"HiSparse speculative swap requires 2-4 steps, got {num_steps}.\"\n        )\n    hot_buffer_size = state.cache_policy.size(1)\n    page_size = device_buffer_tokens.size(1) - hot_buffer_size\n    item_size_bytes = host_cache.stride(0) * host_cache.element_size()\n    record_miss_plan = miss_src is not None\n    if record_miss_plan:\n        if miss_dst is None or miss_count is None:\n            raise ValueError(\n                \"miss_src, miss_dst, and miss_count must be provided together.\"\n            )\n        if miss_src.dtype != torch.int64 or miss_dst.dtype != torch.int32:\n            raise ValueError(\"miss_src must be int64 and miss_dst must be int32.\")\n        if miss_count.dtype != torch.int32:\n            raise ValueError(\"miss_count must be int32.\")\n        plan_capacity = num_steps * num_top_k\n        batch_size = top_k_tokens.size(0)\n        if (","sourceCodeStart":78,"sourceCodeEnd":114,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kvcache/hisparse.py#L78-L114","documentation":"The HiSparse speculative-decode cache swap kernel is compiled for a fixed speculative step count between 2 and 4; top_k_tokens.shape[1] (num_steps) outside that range has no kernel instantiation. The check reads the step dimension of the top-k tensor before launching.","triggerScenarios":"Calling load_cache_to_device_buffer_spec_mla with top_k_tokens of shape [batch, num_steps, top_k] where num_steps is 1 or >4 — e.g. speculative_draft_num_steps misconfigured outside 2..4.","commonSituations":"Setting --speculative-num-steps to 1 (use single-step API instead) or 5+; EAGLE/MTP configs with unsupported step counts; feeding a single-step top-k tensor into the spec API.","solutions":["Set the speculative step count to 2, 3, or 4","For num_steps == 1 use the single-step HiSparse kernel (non-spec API)","Verify top_k_tokens actually has [batch, steps, top_k] layout and you are not reading a different dim as steps"],"exampleFix":"# before (num_steps=1)\ntop_k_tokens = torch.zeros(bs, 1, topk, dtype=torch.int64)\nload_cache_to_device_buffer_spec_mla(...)\n# after (use single-step API)\nload_cache_to_device_buffer_mla(...)  # non-spec variant","handlingStrategy":"validation","validationCode":"_, num_steps, _ = top_k_tokens.shape\nassert 2 <= num_steps <= 4, num_steps","typeGuard":"def spec_steps_supported(top_k_tokens: torch.Tensor) -> bool:\n    return 2 <= top_k_tokens.shape[1] <= 4","tryCatchPattern":"try:\n    load_cache_to_device_buffer_spec_mla(...)\nexcept ValueError:\n    load_cache_to_device_buffer_mla(...)  # single-step fallback","preventionTips":["Configure speculative num_steps in 2..4","Branch to the single-step API when steps == 1"],"tags":["hisparse","speculative-decoding","shape-validation"],"backgroundTag":"value-out-of-supported-range","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}