{"record":{"id":"3fb4b492f43e7c11","repo":"sgl-project/sglang","slug":"query-heads-q-heads-not-divisible-by-kv-heads-k","errorCode":null,"errorMessage":"Query heads {q_heads} not divisible by KV heads {kv_heads}","messagePattern":"Query heads (.+?) not divisible by KV heads (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/sparsity/algorithms/quest_algorithm.py","lineNumber":152,"sourceCode":"        head_dim = k_min.shape[-1]\n        if queries.dim() == 2:\n            bs, hidden = queries.shape\n            if hidden % head_dim != 0:\n                raise ValueError(\n                    f\"Quest query hidden size {hidden} not divisible by head_dim {head_dim}\"\n                )\n            q_heads = hidden // head_dim\n            q = queries.view(bs, q_heads, head_dim)\n        elif queries.dim() == 3:\n            q = queries\n        else:\n            raise ValueError(f\"Unsupported query shape for Quest: {queries.shape}\")\n\n        kv_heads = k_min.shape[-2]\n        q_heads = q.shape[1]\n        if q_heads != kv_heads:\n            if q_heads % kv_heads != 0:\n                raise ValueError(\n                    f\"Query heads {q_heads} not divisible by KV heads {kv_heads}\"\n                )\n            group = q_heads // kv_heads\n            # Average grouped query heads to align with KV heads (approximation for MQA/GQA).\n            q = q.view(q.shape[0], kv_heads, group, head_dim).mean(dim=2)\n\n        q = q.to(k_min.dtype).unsqueeze(1)  # [bs, 1, kv_heads, head_dim]\n\n        criticality = torch.where(q >= 0, q * k_max, q * k_min).sum(dim=(2, 3))\n        criticality = torch.where(\n            valid_mask, criticality, torch.full_like(criticality, float(\"-inf\"))\n        )\n\n        return criticality\n","sourceCodeStart":134,"sourceCodeEnd":167,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/sparsity/algorithms/quest_algorithm.py#L134-L167","documentation":"When query heads outnumber KV heads in Quest retrieval, q_heads must be divisible by kv_heads so grouped heads can be averaged (GQA/MQA alignment). If not divisible, the grouping is invalid and page scores cannot be computed via the mean approximation.","triggerScenarios":"Calling Quest retrieval with q_heads % kv_heads != 0 — e.g. 12 query heads against 8 KV heads, or a query tensor whose head count came from a different GQA group size than the KV cache.","commonSituations":"Custom models with unusual GQA ratios; queries built from an intermediate projection with an arbitrary head count; KV cache configured with a different kv_head count than the model (e.g. --kv-cache-quant or tensor-parallel changes to head counts).","solutions":["Make the query tensor's head count a multiple of the KV cache head count (align GQA group size)","Reproduce MQA by repeating/averaging queries to exactly kv_heads before retrieval","Check TP settings: kv_heads after sharding must divide q_heads after sharding"],"exampleFix":"# before\n# q: (bs, 12, hd), kv cache has 8 heads -> error\n# after\n# align GQA: build q with 8 heads (repeat-interleave averaged) or fix kv_heads=6/12","handlingStrategy":"validation","validationCode":"kv_heads = k_min.shape[-2]\nq_heads = queries.shape[1]\nif q_heads % kv_heads != 0:\n    raise ValueError(f\"{q_heads} q_heads not divisible by {kv_heads} kv_heads\")","typeGuard":"def gqa_aligned(q_heads: int, kv_heads: int) -> bool:\n    return q_heads == kv_heads or q_heads % kv_heads == 0","tryCatchPattern":null,"preventionTips":["Verify TP sharding keeps q_heads a multiple of kv_heads","Test with the production GQA ratio"],"tags":["quest","sparse-attention","gqa","head-mismatch","value-error"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}