{"record":{"id":"17b22cc52078f0b8","repo":"sgl-project/sglang","slug":"q-shape-must-be-num-tokens-num-qo-heads-head-di","errorCode":null,"errorMessage":"q shape must be [num_tokens, num_qo_heads, head_dim], got {q.shape}","messagePattern":"q shape must be \\[num_tokens, num_qo_heads, head_dim\\], got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/aot/python/sgl_kernel/metal.py","lineNumber":78,"sourceCode":"    Returns:\n        `(q_rot, k_rot, k_pool_new, v_pool_new)`.\n    \"\"\"\n    if q.ndim != 3 or k.ndim != 3 or v.ndim != 3:\n        raise ValueError(\"rope_pool_fused expects q/k/v to be 3-D\")\n    if positions.ndim != 1 or slots.ndim != 1:\n        raise ValueError(\"rope_pool_fused expects positions/slots to be 1-D\")\n    if k_pool.ndim != 3 or v_pool.ndim != 3:\n        raise ValueError(\"rope_pool_fused expects pool tensors to be 3-D\")\n    q_shape = tuple(q.shape)\n    k_shape = tuple(k.shape)\n    v_shape = tuple(v.shape)\n    positions_shape = tuple(positions.shape)\n    slots_shape = tuple(slots.shape)\n    k_pool_shape = tuple(k_pool.shape)\n    v_pool_shape = tuple(v_pool.shape)\n\n    if q_shape != (q_shape[0], num_qo_heads, head_dim):\n        raise ValueError(\n            \"q shape must be [num_tokens, num_qo_heads, head_dim], \" f\"got {q.shape}\"\n        )\n    if k_shape != (q_shape[0], num_kv_heads, head_dim):\n        raise ValueError(\n            \"k shape must be [num_tokens, num_kv_heads, head_dim], \" f\"got {k.shape}\"\n        )\n    if v_shape != k_shape:\n        raise ValueError(f\"v shape must match k shape, got {v.shape} vs {k.shape}\")\n    if positions_shape != (q_shape[0],) or slots_shape != (q_shape[0],):\n        raise ValueError(\"positions/slots must have one entry per token\")\n    if k_pool_shape[1:] != (num_kv_heads, head_dim):\n        raise ValueError(f\"k_pool has incompatible shape {k_pool.shape}\")\n    if v_pool_shape != k_pool_shape:\n        raise ValueError(\n            f\"v_pool shape must match k_pool shape, got {v_pool.shape} vs {k_pool.shape}\"\n        )\n    if q.dtype != k.dtype or q.dtype != v.dtype:\n        raise ValueError(\"q/k/v dtypes must match\")","sourceCodeStart":60,"sourceCodeEnd":96,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/aot/python/sgl_kernel/metal.py#L60-L96","documentation":"After rank checks, rope_pool_fused validates the exact shape of q: it must be [num_tokens, num_qo_heads, head_dim] matching the num_qo_heads and head_dim arguments. This error means q's trailing dimensions disagree with the declared head configuration (the leading dim is always accepted, so the mismatch is in heads or head_dim).","triggerScenarios":"Calling rope_pool_fused with num_qo_heads/head_dim arguments that don't match q.shape[1] / q.shape[2], e.g. passing GQA head counts for query heads, or a head_dim from a different model config.","commonSituations":"Copying config values from a different model (head_dim mismatch); mixing num_qo_heads and num_kv_heads argument order; model refactor where q projection output width changed but call site was not updated.","solutions":["Check q.shape and ensure num_qo_heads == q.shape[1] and head_dim == q.shape[2]","If q is packed as [num_tokens, num_qo_heads*head_dim], reshape: q = q.view(num_tokens, num_qo_heads, head_dim)","Verify you did not swap the num_qo_heads and num_kv_heads parameters"],"exampleFix":"# before\nmetal.rope_pool_fused(q, k, v, pos, slots, kp, vp,\n    num_qo_heads=num_kv_heads, num_kv_heads=num_qo_heads, head_dim=head_dim)\n\n# after\nmetal.rope_pool_fused(q, k, v, pos, slots, kp, vp,\n    num_qo_heads=num_qo_heads, num_kv_heads=num_kv_heads, head_dim=head_dim)","handlingStrategy":"validation","validationCode":"assert q.shape[1:] == (num_qo_heads, head_dim), (q.shape, num_qo_heads, head_dim)","typeGuard":"def q_shape_ok(q, num_qo_heads, head_dim):\n    return q.ndim == 3 and q.shape[1] == num_qo_heads and q.shape[2] == head_dim","tryCatchPattern":null,"preventionTips":["Derive num_qo_heads/head_dim from the tensor itself or model config, never hardcode","Use keyword arguments to avoid swapping head-count parameters"],"tags":["shape-validation","rope","gqa","metal","sgl-kernel"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}