{"record":{"id":"139f0942c1249fe9","repo":"sgl-project/sglang","slug":"kv-scales-shape-tuple-kv-scales-shape-does-not","errorCode":null,"errorMessage":"kv_scales shape {tuple(kv_scales.shape)} does not match expected ({unified_kv.shape[0]}, {expected_g})","messagePattern":"kv_scales shape (.+?) does not match expected \\((.+?), (.+?)\\)","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/dsv4/unified_kv_kernels/paged_decode.py","lineNumber":676,"sourceCode":"        )\n\n    quant_kv = kv_scales is not None\n    if quant_kv:\n        if unified_kv.dtype != _FP8_DTYPE:\n            raise RuntimeError(\n                f\"kv_scales supplied but unified_kv is {unified_kv.dtype}, \"\n                f\"expected {_FP8_DTYPE}\"\n            )\n        if kv_scales.dtype != torch.float32:\n            raise RuntimeError(f\"kv_scales must be fp32, got {kv_scales.dtype}\")\n        D_check = unified_kv.shape[-1]\n        if D_check % _FP8_GROUP_SIZE != 0:\n            raise RuntimeError(\n                f\"D={D_check} must be divisible by GROUP_SIZE={_FP8_GROUP_SIZE}\"\n            )\n        expected_g = D_check // _FP8_GROUP_SIZE\n        if kv_scales.shape != (unified_kv.shape[0], expected_g):\n            raise RuntimeError(\n                f\"kv_scales shape {tuple(kv_scales.shape)} does not match \"\n                f\"expected ({unified_kv.shape[0]}, {expected_g})\"\n            )\n        if kv_scales.stride(-1) != 1:\n            kv_scales = kv_scales.contiguous()\n    else:\n        if unified_kv.dtype != q.dtype:\n            raise RuntimeError(\n                f\"unified_kv dtype mismatch: kv={unified_kv.dtype}, q={q.dtype}\"\n            )\n\n    T, H, D = q.shape\n    out = torch.empty_like(q)\n\n    if block_h is None:\n        block_h = triton.next_power_of_2(min(H, 64))\n    else:\n        block_h = triton.next_power_of_2(block_h)","sourceCodeStart":658,"sourceCodeEnd":694,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/dsv4/unified_kv_kernels/paged_decode.py#L658-L694","documentation":"Given unified_kv of shape (num_pages, ..., D), the kernel requires kv_scales to be exactly (unified_kv.shape[0], D // _FP8_GROUP_SIZE): one row of per-group scales per page. Any other shape (wrong page count, wrong group count, flat vector) is rejected.","triggerScenarios":"Passing kv_scales with shape (num_tokens, groups) instead of (num_pages, groups), or scales computed for a different cache size / head_dim than the current unified_kv.","commonSituations":"Reusing a scales tensor after the paged cache grew (pages appended); mismatch between the quantization routine's page layout and the kernel's expectation; off-by-one in group count after a head_dim change.","solutions":["Recompute/requantize kv_scales against the current unified_kv so its shape is (num_pages, D // _FP8_GROUP_SIZE)","Check that the page dimension used when quantizing matches unified_kv.shape[0]","Ensure the cache and scales are produced by the same allocation/quantization step, not independently"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if kv_scales is not None:\n    expected = (unified_kv.shape[0], unified_kv.shape[-1] // FP8_GROUP_SIZE)\n    assert tuple(kv_scales.shape) == expected, (kv_scales.shape, expected)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Requantize scales whenever the paged cache is resized","Treat cache and scales as one unit: allocate/grow them together"],"tags":["fp8","kv-cache","shape-validation","scales"],"backgroundTag":"shape-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}