{"record":{"id":"a8dea6ae160414ee","repo":"sgl-project/sglang","slug":"miss-count-must-be-int32","errorCode":null,"errorMessage":"miss_count must be int32.","messagePattern":"miss_count must be int32\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kvcache/hisparse.py","lineNumber":111,"sourceCode":"    \"\"\"\n    _, num_steps, num_top_k = top_k_tokens.shape\n    if not 2 <= num_steps <= 4:\n        raise ValueError(\n            f\"HiSparse speculative swap requires 2-4 steps, got {num_steps}.\"\n        )\n    hot_buffer_size = state.cache_policy.size(1)\n    page_size = device_buffer_tokens.size(1) - hot_buffer_size\n    item_size_bytes = host_cache.stride(0) * host_cache.element_size()\n    record_miss_plan = miss_src is not None\n    if record_miss_plan:\n        if miss_dst is None or miss_count is None:\n            raise ValueError(\n                \"miss_src, miss_dst, and miss_count must be provided together.\"\n            )\n        if miss_src.dtype != torch.int64 or miss_dst.dtype != torch.int32:\n            raise ValueError(\"miss_src must be int64 and miss_dst must be int32.\")\n        if miss_count.dtype != torch.int32:\n            raise ValueError(\"miss_count must be int32.\")\n        plan_capacity = num_steps * num_top_k\n        batch_size = top_k_tokens.size(0)\n        if (\n            miss_src.ndim != 2\n            or miss_dst.ndim != 2\n            or miss_src.size(0) < batch_size\n            or miss_dst.size(0) < batch_size\n            or miss_src.size(1) < plan_capacity\n            or miss_dst.size(1) < plan_capacity\n        ):\n            raise ValueError(\n                \"speculative miss_src/miss_dst must have shape \"\n                f\"[batch, >= steps * top_k] (capacity {plan_capacity}).\"\n            )\n        if miss_count.ndim != 1 or miss_count.numel() < batch_size:\n            raise ValueError(\"speculative miss_count must have shape [batch].\")\n        if miss_src.stride(0) != miss_dst.stride(0):\n            raise ValueError(\"miss_src/miss_dst row strides must match.\")","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kvcache/hisparse.py#L93-L129","documentation":"The third dtype check in the miss-plan triple: miss_count must be int32 because the kernel writes per-batch miss counts as 32-bit values. int64 or int16 counters are rejected to avoid ABI mismatch on kernel write-back.","triggerScenarios":"Calling load_cache_to_device_buffer_spec_mla with a miss-plan and miss_count.dtype != torch.int32 (commonly torch.int64).","commonSituations":"Reusing an int64 counter tensor from elsewhere in the pipeline; allocating all plan tensors as int64 for uniformity.","solutions":["Allocate miss_count with dtype=torch.int32","If you track counts in int64 elsewhere, add a separate int32 staging tensor for the kernel"],"exampleFix":"# before\nmiss_count = torch.zeros(bs, dtype=torch.int64, device=dev)\n# after\nmiss_count = torch.zeros(bs, dtype=torch.int32, device=dev)","handlingStrategy":"type-guard","validationCode":"assert miss_count.dtype == torch.int32","typeGuard":"def miss_count_dtype_ok(c) -> bool:\n    return c is None or c.dtype == torch.int32","tryCatchPattern":null,"preventionTips":["Allocate counters as int32; keep a separate int64 tensor if you need wide counts elsewhere"],"tags":["hisparse","dtype-validation","miss-plan"],"backgroundTag":"dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}