sgl-project/sglang · error · ValueError

Quest query hidden size {hidden} not divisible by head_dim {

Error message

Quest query hidden size {hidden} not divisible by head_dim {head_dim}

What it means

In Quest sparse retrieval, when queries are 2-D (bs, hidden), the hidden dimension must be divisible by the KV head_dim (from k_min) so it can be reshaped to (bs, q_heads, head_dim). If not, the query layout doesn't match the cached page metadata and scores cannot be computed.

Source

Thrown at python/sglang/srt/mem_cache/sparsity/algorithms/quest_algorithm.py:138

    def _retrieve_page_scores(
        self,
        layer_id: int,
        phys_pages: torch.Tensor,
        req_pool_indices: torch.Tensor,
        queries: torch.Tensor,
    ) -> torch.Tensor:
        # Clamp pages to valid storage range
        phys_pages_clamped = phys_pages.clamp(0, self.page_k_min[layer_id].shape[0] - 1)

        k_min = self.page_k_min[layer_id][phys_pages_clamped]
        k_max = self.page_k_max[layer_id][phys_pages_clamped]
        valid_mask = self.page_valid[layer_id][phys_pages_clamped]
        # Align query shape to KV heads.
        head_dim = k_min.shape[-1]
        if queries.dim() == 2:
            bs, hidden = queries.shape
            if hidden % head_dim != 0:
                raise ValueError(
                    f"Quest query hidden size {hidden} not divisible by head_dim {head_dim}"
                )
            q_heads = hidden // head_dim
            q = queries.view(bs, q_heads, head_dim)
        elif queries.dim() == 3:
            q = queries
        else:
            raise ValueError(f"Unsupported query shape for Quest: {queries.shape}")

        kv_heads = k_min.shape[-2]
        q_heads = q.shape[1]
        if q_heads != kv_heads:
            if q_heads % kv_heads != 0:
                raise ValueError(
                    f"Query heads {q_heads} not divisible by KV heads {kv_heads}"
                )
            group = q_heads // kv_heads
            # Average grouped query heads to align with KV heads (approximation for MQA/GQA).

View on GitHub (pinned to 0132848349)

Solutions

1. Pass 3-D queries shaped (bs, q_heads, head_dim) if head dims differ so the algorithm can handle GQA alignment explicitly
2. Project queries to kv head_dim before retrieval (apply the model's qk projection / head_dim reshaping)
3. Fix attention config so query head_dim equals the KV cache head_dim used by Quest pages

Example fix

# before
scores = quest._retrieve_page_scores(queries=q_flat, ...)  # (bs, hidden), hidden % head_dim != 0
# after
q = q_flat.view(bs, q_heads, head_dim)
scores = quest._retrieve_page_scores(queries=q, ...)

head_dim = k_min.shape[-1]
if queries.dim() == 2 and queries.shape[-1] % head_dim != 0:
    queries = queries.view(bs, -1, head_dim) if queries.shape[-1] % head_dim == 0 else project(queries)
# or simply pass 3-D queries

def quest_query_ok(queries, head_dim: int) -> bool:
    return queries.dim() == 3 or (queries.dim() == 2 and queries.shape[-1] % head_dim == 0)

When it happens

Trigger: Calling Quest _retrieve_page_scores with a 2-D query tensor whose last dim (e.g. q_heads*head_dim from a different head_dim config) is not a multiple of the k cache's head_dim — typically a mismatch between model query head_dim and KV cache head_dim (e.g. 128 vs 64 without proper projection).

Common situations: Configuring Quest with a model whose query head_dim differs from the KV head_dim and no GQA projection applied; feeding flattened queries from a different layer shape; mixing 2-D flattened input where 3-D was expected.

Related errors

• Unsupported query shape for Quest: {queries.shape}
• Query heads {q_heads} not divisible by KV heads {kv_heads}
• kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}
• indices must have shape ({s_q}, {h_kv}, topk), got {tuple(in
• forward_batch with seq_lens is required for TopK retrieval

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6cbdee944a3bb6f5. Report an issue: GitHub.