{"record":{"id":"4c3661172ab726a4","repo":"jax-ml/jax","slug":"q-head-dim-head-dim-must-be-the-same-as-that-of","errorCode":null,"errorMessage":"Q head_dim {head_dim} must be the same as that of K/V {head_dim_k}.","messagePattern":"Q head_dim (.+?) must be the same as that of K/V (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"jax/experimental/pallas/ops/tpu/ragged_paged_attention/kernel.py","lineNumber":246,"sourceCode":"    mask_value: float | None = None,\n    k_scale: float | None = None,\n    v_scale: float | None = None,\n    # Kernel tuning params.\n    num_kv_pages_per_block: int | None = None,\n    num_queries_per_block: int | None = None,\n    vmem_limit_bytes: int | None = None,\n):\n  _, num_q_heads, head_dim = q.shape\n  _, _, num_combined_kv_heads, head_dim_k = kv_pages.shape\n  assert num_combined_kv_heads % 2 == 0\n  assert isinstance(k_scale, float) or k_scale is None\n  assert isinstance(v_scale, float) or v_scale is None\n  num_kv_heads = num_combined_kv_heads // 2\n  max_num_seqs, pages_per_seq = page_indices.shape\n  if num_seqs.shape != (1,):\n    raise ValueError(f\"{num_seqs.shape=} must be (1,)\")\n  if head_dim_k != head_dim:\n    raise ValueError(\n        f\"Q head_dim {head_dim} must be the same as that of K/V {head_dim_k}.\"\n    )\n  if kv_lens.shape != (max_num_seqs,):\n    raise ValueError(\n        f\"Expected {kv_lens.shape=} to be ({max_num_seqs},) where\"\n        \" `max_num_seqs` is `page_indices.shape[0]`.\"\n    )\n  if cu_q_lens.shape != (max_num_seqs + 1,):\n    raise ValueError(\n        f\"Expected {cu_q_lens.shape=} to be ({max_num_seqs + 1},)  where\"\n        \" `max_num_seqs` is `page_indices.shape[0]`.\"\n    )\n  if (\n      kv_lens.dtype != jnp.int32\n      or page_indices.dtype != jnp.int32\n      or cu_q_lens.dtype != jnp.int32\n  ):\n    raise ValueError(","sourceCodeStart":228,"sourceCodeEnd":264,"githubUrl":"https://github.com/jax-ml/jax/blob/1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb/jax/experimental/pallas/ops/tpu/ragged_paged_attention/kernel.py#L228-L264","documentation":"Static validation of ragged paged attention checks that q's head_dim equals the head_dim of the combined K/V pages tensor (which packs K and V along the head axis). Since each query is dotted with K/V of the same dimensionality, a mismatch is a hard compile-time error.","triggerScenarios":"Calling ragged_paged_attention with q.shape[-1]=128 but kv_pages built with head_dim=64; loading a KV cache saved from a model with a different head dimension; mis-stacking K and V so the combined tensor's last dim no longer matches.","commonSituations":"Model swaps reusing a persisted paged KV cache; GQA refactor changing head_dim; building kv_pages by stacking K/V with an incorrect axis order.","solutions":["Rebuild kv_pages with head_dim equal to q.shape[-1]","Project q to the cache's head_dim before calling","Always allocate the cache from the model config's head_dim, not a hardcoded value"],"exampleFix":"// before\nkv_pages = stack_cache(k_heads_64, v_heads_64)  # q head_dim 128\n// after\nkv_pages = stack_cache(k_heads_128, v_heads_128)  # matches q","handlingStrategy":"validation","validationCode":"head_dim_k = kv_pages.shape[-1]  # combined K/V pages tensor\nassert q.shape[-1] == head_dim_k","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Allocate the paged cache from the live model config's head_dim","Discard persisted caches when switching models or head_dim-affecting settings"],"tags":["jax","pallas","tpu","ragged-attention","head-dim-validation"],"backgroundTag":"shape-validation-failed","analyzedSha":"1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb","analyzedAt":"2026-08-27T09:53:25.647Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}