{"record":{"id":"6cbdee944a3bb6f5","repo":"sgl-project/sglang","slug":"quest-query-hidden-size-hidden-not-divisible-by","errorCode":null,"errorMessage":"Quest query hidden size {hidden} not divisible by head_dim {head_dim}","messagePattern":"Quest query hidden size (.+?) not divisible by head_dim (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/sparsity/algorithms/quest_algorithm.py","lineNumber":138,"sourceCode":"    def _retrieve_page_scores(\n        self,\n        layer_id: int,\n        phys_pages: torch.Tensor,\n        req_pool_indices: torch.Tensor,\n        queries: torch.Tensor,\n    ) -> torch.Tensor:\n        # Clamp pages to valid storage range\n        phys_pages_clamped = phys_pages.clamp(0, self.page_k_min[layer_id].shape[0] - 1)\n\n        k_min = self.page_k_min[layer_id][phys_pages_clamped]\n        k_max = self.page_k_max[layer_id][phys_pages_clamped]\n        valid_mask = self.page_valid[layer_id][phys_pages_clamped]\n        # Align query shape to KV heads.\n        head_dim = k_min.shape[-1]\n        if queries.dim() == 2:\n            bs, hidden = queries.shape\n            if hidden % head_dim != 0:\n                raise ValueError(\n                    f\"Quest query hidden size {hidden} not divisible by head_dim {head_dim}\"\n                )\n            q_heads = hidden // head_dim\n            q = queries.view(bs, q_heads, head_dim)\n        elif queries.dim() == 3:\n            q = queries\n        else:\n            raise ValueError(f\"Unsupported query shape for Quest: {queries.shape}\")\n\n        kv_heads = k_min.shape[-2]\n        q_heads = q.shape[1]\n        if q_heads != kv_heads:\n            if q_heads % kv_heads != 0:\n                raise ValueError(\n                    f\"Query heads {q_heads} not divisible by KV heads {kv_heads}\"\n                )\n            group = q_heads // kv_heads\n            # Average grouped query heads to align with KV heads (approximation for MQA/GQA).","sourceCodeStart":120,"sourceCodeEnd":156,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/sparsity/algorithms/quest_algorithm.py#L120-L156","documentation":"In Quest sparse retrieval, when queries are 2-D (bs, hidden), the hidden dimension must be divisible by the KV head_dim (from k_min) so it can be reshaped to (bs, q_heads, head_dim). If not, the query layout doesn't match the cached page metadata and scores cannot be computed.","triggerScenarios":"Calling Quest _retrieve_page_scores with a 2-D query tensor whose last dim (e.g. q_heads*head_dim from a different head_dim config) is not a multiple of the k cache's head_dim — typically a mismatch between model query head_dim and KV cache head_dim (e.g. 128 vs 64 without proper projection).","commonSituations":"Configuring Quest with a model whose query head_dim differs from the KV head_dim and no GQA projection applied; feeding flattened queries from a different layer shape; mixing 2-D flattened input where 3-D was expected.","solutions":["Pass 3-D queries shaped (bs, q_heads, head_dim) if head dims differ so the algorithm can handle GQA alignment explicitly","Project queries to kv head_dim before retrieval (apply the model's qk projection / head_dim reshaping)","Fix attention config so query head_dim equals the KV cache head_dim used by Quest pages"],"exampleFix":"# before\nscores = quest._retrieve_page_scores(queries=q_flat, ...)  # (bs, hidden), hidden % head_dim != 0\n# after\nq = q_flat.view(bs, q_heads, head_dim)\nscores = quest._retrieve_page_scores(queries=q, ...)","handlingStrategy":"validation","validationCode":"head_dim = k_min.shape[-1]\nif queries.dim() == 2 and queries.shape[-1] % head_dim != 0:\n    queries = queries.view(bs, -1, head_dim) if queries.shape[-1] % head_dim == 0 else project(queries)\n# or simply pass 3-D queries","typeGuard":"def quest_query_ok(queries, head_dim: int) -> bool:\n    return queries.dim() == 3 or (queries.dim() == 2 and queries.shape[-1] % head_dim == 0)","tryCatchPattern":null,"preventionTips":["Pass 3-D (bs, heads, head_dim) queries","Ensure model q head_dim matches KV cache head_dim"],"tags":["quest","sparse-attention","head-dim","shape-mismatch","value-error"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}