{"record":{"id":"a8155e3f08886f72","repo":"sgl-project/sglang","slug":"unsupported-query-shape-for-quest-queries-shape","errorCode":null,"errorMessage":"Unsupported query shape for Quest: {queries.shape}","messagePattern":"Unsupported query shape for Quest: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/sparsity/algorithms/quest_algorithm.py","lineNumber":146,"sourceCode":"        phys_pages_clamped = phys_pages.clamp(0, self.page_k_min[layer_id].shape[0] - 1)\n\n        k_min = self.page_k_min[layer_id][phys_pages_clamped]\n        k_max = self.page_k_max[layer_id][phys_pages_clamped]\n        valid_mask = self.page_valid[layer_id][phys_pages_clamped]\n        # Align query shape to KV heads.\n        head_dim = k_min.shape[-1]\n        if queries.dim() == 2:\n            bs, hidden = queries.shape\n            if hidden % head_dim != 0:\n                raise ValueError(\n                    f\"Quest query hidden size {hidden} not divisible by head_dim {head_dim}\"\n                )\n            q_heads = hidden // head_dim\n            q = queries.view(bs, q_heads, head_dim)\n        elif queries.dim() == 3:\n            q = queries\n        else:\n            raise ValueError(f\"Unsupported query shape for Quest: {queries.shape}\")\n\n        kv_heads = k_min.shape[-2]\n        q_heads = q.shape[1]\n        if q_heads != kv_heads:\n            if q_heads % kv_heads != 0:\n                raise ValueError(\n                    f\"Query heads {q_heads} not divisible by KV heads {kv_heads}\"\n                )\n            group = q_heads // kv_heads\n            # Average grouped query heads to align with KV heads (approximation for MQA/GQA).\n            q = q.view(q.shape[0], kv_heads, group, head_dim).mean(dim=2)\n\n        q = q.to(k_min.dtype).unsqueeze(1)  # [bs, 1, kv_heads, head_dim]\n\n        criticality = torch.where(q >= 0, q * k_max, q * k_min).sum(dim=(2, 3))\n        criticality = torch.where(\n            valid_mask, criticality, torch.full_like(criticality, float(\"-inf\"))\n        )","sourceCodeStart":128,"sourceCodeEnd":164,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/sparsity/algorithms/quest_algorithm.py#L128-L164","documentation":"Quest's _retrieve_page_scores only accepts 1-D-projected 2-D or 3-D query tensors; anything else (0-D, 4-D, etc.) raises ValueError with the offending shape. The algorithm must reshape or use queries directly as (bs, q_heads, head_dim).","triggerScenarios":"Passing queries with dim() not in (2, 3) — e.g. a 4-D (bs, layers, heads, dim) tensor or a 1-D flattened vector — to Quest retrieval.","commonSituations":"Adapter code forwarding raw multi-dimensional attention tensors straight into the sparse algorithm instead of the per-layer per-request queries.","solutions":["Squeeze/reshape queries to (bs, hidden) or (bs, q_heads, head_dim) before calling retrieve","Index the specific layer: queries = full_q[:, layer_id] before retrieval"],"exampleFix":"# before\nscores = quest._retrieve_page_scores(queries=q_all_layers, ...)  # 4-D\n# after\nscores = quest._retrieve_page_scores(queries=q_all_layers[:, layer_id], ...)  # 3-D","handlingStrategy":"type-guard","validationCode":"assert queries.dim() in (2, 3), f\"bad query rank: {queries.shape}\"","typeGuard":"def quest_rank_ok(queries) -> bool:\n    return queries.dim() in (2, 3)","tryCatchPattern":null,"preventionTips":["Reshape before retrieval; never forward multi-layer tensors unindexed"],"tags":["quest","sparse-attention","tensor-rank","shape-mismatch","value-error"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}