{"record":{"id":"7b5234165073ce2e","repo":"Comfy-Org/ComfyUI","slug":"query-heads-must-be-divisible-by-key-value-heads-f","errorCode":null,"errorMessage":"Query heads must be divisible by key/value heads for GQA: {query_heads} vs {key_heads}","messagePattern":"Query heads must be divisible by key/value heads for GQA: (.+?) vs (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"comfy/ops.py","lineNumber":46,"sourceCode":"import comfy.pinned_memory\nimport comfy.utils\n\nimport comfy_aimdo.model_vbar\nimport comfy_aimdo.torch\n\ndef run_every_op():\n    if torch.compiler.is_compiling():\n        return\n\n    comfy.model_management.throw_exception_if_processing_interrupted()\n\ndef gqa_repeat_factor(query_heads, key_heads, value_heads):\n    if key_heads != value_heads:\n        raise ValueError(f\"Key/value head count mismatch for GQA: {key_heads} != {value_heads}\")\n    if query_heads == key_heads:\n        return 1\n    if query_heads % key_heads != 0:\n        raise ValueError(f\"Query heads must be divisible by key/value heads for GQA: {query_heads} vs {key_heads}\")\n    return query_heads // key_heads\n\ndef repeat_kv_for_gqa(k, v, query_heads, head_dim):\n    n_rep = gqa_repeat_factor(query_heads, k.shape[head_dim], v.shape[head_dim])\n    if n_rep > 1:\n        k = k.repeat_interleave(n_rep, dim=head_dim)\n        v = v.repeat_interleave(n_rep, dim=head_dim)\n    return k, v\n\ndef scaled_dot_product_attention(q, k, v, *args, **kwargs):\n    attn_mask = args[0] if len(args) > 0 else kwargs.get(\"attn_mask\")\n    if kwargs.get(\"enable_gqa\", False) and attn_mask is not None:\n        k, v = repeat_kv_for_gqa(k, v, q.shape[-3], -3)\n        kwargs[\"enable_gqa\"] = False\n    return torch.nn.functional.scaled_dot_product_attention(q, k, v, *args, **kwargs)\n\n\ntry:","sourceCodeStart":28,"sourceCodeEnd":64,"githubUrl":"https://github.com/Comfy-Org/ComfyUI/blob/1c6d8d45b3693bfbb32385b410d813a7fd6be216/comfy/ops.py#L28-L64","documentation":"For grouped-query attention the query head count must be an exact multiple of the key/value head count, otherwise K/V heads cannot be evenly repeated to match Q. gqa_repeat_factor raises when query_heads % key_heads != 0, after first checking K/V equality. Typical configs are 8Q/2KV or 32Q/8KV; anything non-divisible is a config or weight-layout error.","triggerScenarios":"repeat_kv_for_gqa(k, v, query_heads=12, head_dim=...) with 8 KV heads; or passing the sequence length / head dim as query_heads by mistake.","commonSituations":"Custom attention configs with non-divisible head splits; passing num_kv_heads where query heads are expected; checkpoints with unusual GQA ratios that need a different attention path.","solutions":["Make query_heads a multiple of the KV head count (e.g. 8/2, 16/4, 32/8).","Verify the argument order — query_heads is the full query head count, not the group size.","If the model genuinely uses MQA (1 KV head) ensure KV tensors actually have shape 1 on the head axis."],"exampleFix":"# before\nk, v = repeat_kv_for_gqa(k, v, query_heads=12, head_dim=2)  # kv heads = 8\n# after\nk, v = repeat_kv_for_gqa(k, v, query_heads=16, head_dim=2)  # 16 % 8 == 0","handlingStrategy":"validation","validationCode":"kv_heads = k.shape[head_dim]\nif q_heads % kv_heads != 0:\n    raise ValueError(f\"q_heads {q_heads} not divisible by kv_heads {kv_heads}; fix attention config\")\nn_rep = gqa_repeat_factor(q_heads, kv_heads, kv_heads)","typeGuard":"def is_valid_gqa(q_heads, kv_heads) -> bool:\n    return q_heads % kv_heads == 0","tryCatchPattern":null,"preventionTips":["Use standard GQA ratios (q_heads an exact multiple of kv_heads).","Pass the full query head count, not the group size, as query_heads."],"tags":["attention","gqa","config","heads"],"backgroundTag":null,"analyzedSha":"1c6d8d45b3693bfbb32385b410d813a7fd6be216","analyzedAt":"2026-08-14T19:37:18.893Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}