{"record":{"id":"3980d98554f92269","repo":"jax-ml/jax","slug":"in-mha-expected-number-of-key-heads-num-kv-he","errorCode":null,"errorMessage":"In MHA, expected number of 'key' heads ({num_kv_heads}) to be a multiple of the number of 'query' heads ({num_q_heads})","messagePattern":"In MHA, expected number of 'key' heads \\((.+?)\\) to be a multiple of the number of 'query' heads \\((.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"jax/experimental/pallas/ops/tpu/splash_attention/splash_attention_kernel.py","lineNumber":956,"sourceCode":"    raise ValueError(\n        \"partial_mask_blocks must be of type np.bool_ but got\"\n        f\" {partial_mask_blocks.dtype}\"\n    )\n\n  if len(k.shape) != expected_kv_rank:\n    raise ValueError(\n        f\"Expected {expected_kv_rank}-dim 'key' tensor for MQA. Instead got a\"\n        f\" {len(k.shape)}-dim one.\"\n    )\n\n  if k.shape[kv_head_dimension] != head_dim_qk:\n    raise ValueError(\n        f\"Expected 'key' head dimension to be: {head_dim_qk}. Instead got:\"\n        f\" {k.shape[kv_head_dimension]}.\"\n    )\n\n  if not is_mqa and num_q_heads % num_kv_heads != 0:\n    raise ValueError(\n        f\"In MHA, expected number of 'key' heads ({num_kv_heads}) to be a\"\n        f\" multiple of the number of 'query' heads ({num_q_heads})\"\n    )\n\n  if k.shape[:-1] != v.shape[:-1]:\n    raise ValueError(\n        f\"Expected 'key' {k.shape} and 'value' {v.shape} to have the same \"\n        \"leading dimensions.\"\n    )\n\n  assert bkv_compute is not None\n  if bkv % bkv_compute:\n    raise ValueError(f\"{bkv=} must be a multiple of {bkv_compute=}.\")\n  if bkv_compute % NUM_LANES:\n    raise ValueError(f\"{bkv_compute=} must be a multiple of {NUM_LANES}.\")\n\n  kv_seq_len = k.shape[kv_seq_len_dimension]\n","sourceCodeStart":938,"sourceCodeEnd":974,"githubUrl":"https://github.com/jax-ml/jax/blob/1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb/jax/experimental/pallas/ops/tpu/splash_attention/splash_attention_kernel.py#L938-L974","documentation":"For multi-head attention (not MQA), the number of query heads must be divisible by the number of KV heads (GQA). Note the message text says 'key heads to be a multiple of query heads' but the check is num_q_heads % num_kv_heads != 0 — i.e. q_heads must be a multiple of kv_heads; otherwise per-KV-head grouping of query heads is impossible.","triggerScenarios":"Passing q with 12 heads and k/v with 8 heads; any head configuration where num_q_heads % num_kv_heads != 0 (e.g. 48 Q heads, 32 KV heads).","commonSituations":"Using GQA ratios from Llama-style models incorrectly (e.g. 8 KV heads is fine for 32 Q heads, but 7 KV heads is not); mixing configs between model and kernel.","solutions":["Make num_q_heads a multiple of num_kv_heads (e.g. 32 Q / 8 KV)","If you truly need unmatched heads, repeat/expand K,V heads: jnp.repeat(k, num_q_heads//num_kv_heads, axis=0) after choosing an integer ratio, or run MHA (equal heads)","Double-check the head axis order — you may have swapped num_heads and num_kv_heads when reshaping"],"exampleFix":"// before\nq: [12, seq, hd], k/v: [8, seq, hd]\n// after\nq: [16, seq, hd], k/v: [8, seq, hd]  # or repeat k,v to 16 heads for MHA","handlingStrategy":"validation","validationCode":"assert q.shape[0] % k.shape[0] == 0, 'num_q_heads must be divisible by num_kv_heads'","typeGuard":"def gqa_ok(q_heads: int, kv_heads: int) -> bool:\n    return q_heads % kv_heads == 0","tryCatchPattern":null,"preventionTips":["Sanity-check GQA ratios in config validation","Print head counts once at model init"],"tags":["jax","pallas","tpu","splash-attention","gqa","shape"],"backgroundTag":"head-count-mismatch","analyzedSha":"1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb","analyzedAt":"2026-08-27T09:53:25.647Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}