{"record":{"id":"f9927a5331c85589","repo":"sgl-project/sglang","slug":"d-d-check-must-be-divisible-by-group-size-fp8","errorCode":null,"errorMessage":"D={D_check} must be divisible by GROUP_SIZE={_FP8_GROUP_SIZE}","messagePattern":"D=(.+?) must be divisible by GROUP_SIZE=(.+?)","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/dsv4/unified_kv_kernels/paged_decode.py","lineNumber":671,"sourceCode":"            \"Triton sparse_attn_v4_paged_decode requires CUDA/HIP tensors\"\n        )\n    if q.dtype not in (torch.bfloat16, torch.float16):\n        raise RuntimeError(\n            f\"sparse_attn_v4_paged_decode expects fp16/bf16 q, got {q.dtype}\"\n        )\n\n    quant_kv = kv_scales is not None\n    if quant_kv:\n        if unified_kv.dtype != _FP8_DTYPE:\n            raise RuntimeError(\n                f\"kv_scales supplied but unified_kv is {unified_kv.dtype}, \"\n                f\"expected {_FP8_DTYPE}\"\n            )\n        if kv_scales.dtype != torch.float32:\n            raise RuntimeError(f\"kv_scales must be fp32, got {kv_scales.dtype}\")\n        D_check = unified_kv.shape[-1]\n        if D_check % _FP8_GROUP_SIZE != 0:\n            raise RuntimeError(\n                f\"D={D_check} must be divisible by GROUP_SIZE={_FP8_GROUP_SIZE}\"\n            )\n        expected_g = D_check // _FP8_GROUP_SIZE\n        if kv_scales.shape != (unified_kv.shape[0], expected_g):\n            raise RuntimeError(\n                f\"kv_scales shape {tuple(kv_scales.shape)} does not match \"\n                f\"expected ({unified_kv.shape[0]}, {expected_g})\"\n            )\n        if kv_scales.stride(-1) != 1:\n            kv_scales = kv_scales.contiguous()\n    else:\n        if unified_kv.dtype != q.dtype:\n            raise RuntimeError(\n                f\"unified_kv dtype mismatch: kv={unified_kv.dtype}, q={q.dtype}\"\n            )\n\n    T, H, D = q.shape\n    out = torch.empty_like(q)","sourceCodeStart":653,"sourceCodeEnd":689,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/dsv4/unified_kv_kernels/paged_decode.py#L653-L689","documentation":"The FP8 group quantization used by sparse_attn_v4_paged_decode quantizes the KV head dimension in fixed-size groups (_FP8_GROUP_SIZE). The last dimension of unified_kv must be divisible by that group size, otherwise per-group scales cannot be laid out.","triggerScenarios":"Calling sparse_attn_v4_paged_decode with a unified_kv whose head_dim D is not divisible by _FP8_GROUP_SIZE while kv_scales is provided.","commonSituations":"A DeepSeek variant with an unusual head_dim (e.g. 192 vs 128) run with FP8 KV quantization enabled; head_dim config mismatch between model and cache.","solutions":["Check the model's head_dim; if it is not divisible by the FP8 group size, disable FP8 KV quantization for this model","Verify the unified_kv cache was built with the intended head_dim (no padding/truncation bug)","Pad or reshape the cache last dim to a multiple of _FP8_GROUP_SIZE if your integration allows"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"FP8_GROUP_SIZE = 128  # keep in sync with kernel\nif kv_scales is not None:\n    assert unified_kv.shape[-1] % FP8_GROUP_SIZE == 0","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate head_dim compatibility with FP8 grouping at model-load time, before any cache allocation","Gate FP8 KV quantization on model head_dim in the config layer"],"tags":["fp8","head-dim","kv-cache","shape-validation"],"backgroundTag":"shape-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}