{"record":{"id":"8ad0653a0e78188b","repo":"sgl-project/sglang","slug":"speculative-miss-src-miss-dst-must-have-shape-bat","errorCode":null,"errorMessage":"speculative miss_src/miss_dst must have shape [batch, >= steps * top_k] (capacity {plan_capacity}).","messagePattern":"speculative miss_src/miss_dst must have shape \\[batch, >= steps \\* top_k\\] \\(capacity (.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kvcache/hisparse.py","lineNumber":122,"sourceCode":"        if miss_dst is None or miss_count is None:\n            raise ValueError(\n                \"miss_src, miss_dst, and miss_count must be provided together.\"\n            )\n        if miss_src.dtype != torch.int64 or miss_dst.dtype != torch.int32:\n            raise ValueError(\"miss_src must be int64 and miss_dst must be int32.\")\n        if miss_count.dtype != torch.int32:\n            raise ValueError(\"miss_count must be int32.\")\n        plan_capacity = num_steps * num_top_k\n        batch_size = top_k_tokens.size(0)\n        if (\n            miss_src.ndim != 2\n            or miss_dst.ndim != 2\n            or miss_src.size(0) < batch_size\n            or miss_dst.size(0) < batch_size\n            or miss_src.size(1) < plan_capacity\n            or miss_dst.size(1) < plan_capacity\n        ):\n            raise ValueError(\n                \"speculative miss_src/miss_dst must have shape \"\n                f\"[batch, >= steps * top_k] (capacity {plan_capacity}).\"\n            )\n        if miss_count.ndim != 1 or miss_count.numel() < batch_size:\n            raise ValueError(\"speculative miss_count must have shape [batch].\")\n        if miss_src.stride(0) != miss_dst.stride(0):\n            raise ValueError(\"miss_src/miss_dst row strides must match.\")\n    else:\n        if miss_dst is not None or miss_count is not None:\n            raise ValueError(\n                \"miss_src, miss_dst, and miss_count must be provided together.\"\n            )\n        empty = torch.empty(0)\n        miss_src = miss_dst = miss_count = empty\n\n    module = _jit_spec_module(\n        item_size_bytes,\n        _GATHER_BLOCK_SIZE,","sourceCodeStart":104,"sourceCodeEnd":140,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kvcache/hisparse.py#L104-L140","documentation":"The miss-plan src/dst buffers must be 2-D with at least batch rows and at least steps*top_k columns, because in the worst case every speculative token of every step misses. The check enforces ndim==2, size(0)>=batch and size(1)>=num_steps*num_top_k (the reported plan_capacity).","triggerScenarios":"Calling load_cache_to_device_buffer_spec_mla with miss_src/miss_dst that are 1-D, have fewer rows than top_k_tokens.size(0), or fewer columns than num_steps*num_top_k (e.g. allocated for the single-step capacity only).","commonSituations":"Reusing single-step miss buffers in the spec path; allocating capacity = top_k instead of steps*top_k; batch-size growth after warmup with preallocated plan tensors.","solutions":["Allocate miss_src/miss_dst as [batch, num_steps * top_k] (or larger), int64/int32 respectively","Grow plan buffers whenever batch size or spec steps/top_k change","Assert shapes before launch in your harness"],"exampleFix":"# before\nmiss_src = torch.zeros(bs, topk, dtype=torch.int64, device=dev)\n# after\ncap = num_steps * topk\nmiss_src = torch.zeros(bs, cap, dtype=torch.int64, device=dev)","handlingStrategy":"validation","validationCode":"batch = top_k_tokens.size(0); cap = num_steps * top_k_tokens.size(2)\nassert miss_src.ndim == 2 and miss_src.size(0) >= batch and miss_src.size(1) >= cap\nassert miss_dst.ndim == 2 and miss_dst.size(0) >= batch and miss_dst.size(1) >= cap","typeGuard":"def miss_plan_capacity_ok(top_k_tokens, miss_src, miss_dst) -> bool:\n    cap = top_k_tokens.shape[1] * top_k_tokens.shape[2]\n    b = top_k_tokens.size(0)\n    return (miss_src.ndim == 2 and miss_src.size(0) >= b and miss_src.size(1) >= cap\n            and miss_dst.ndim == 2 and miss_dst.size(0) >= b and miss_dst.size(1) >= cap)","tryCatchPattern":null,"preventionTips":["Size plan buffers as steps*top_k, not top_k","Reallocate plan tensors whenever steps, top_k, or batch size change"],"tags":["hisparse","shape-validation","miss-plan"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}