{"record":{"id":"d97d5d8528e2b364","repo":"vllm-project/vllm","slug":"the-model-s-number-of-query-heads-per-kv-head-nu","errorCode":null,"errorMessage":"The model's number of query heads per KV head ({num_q_per_kv}) must be divisible by `--decode-context-parallel-size` ({decode_context_parallel_size}) for GQA/MQA.","messagePattern":"The model's number of query heads per KV head \\((.+?)\\) must be divisible by `--decode-context-parallel-size` \\((.+?)\\) for GQA/MQA\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/model.py","lineNumber":1410,"sourceCode":"                    f\"`--tensor-parallel-size` ({tensor_parallel_size}) to be \"\n                    \"greater than the model's total number of KV heads \"\n                    f\"({total_num_kv_heads}). Increase `--tensor-parallel-size` \"\n                    \"or set `--decode-context-parallel-size 1`.\"\n                )\n\n            max_dcp_size = tensor_parallel_size // total_num_kv_heads\n            if decode_context_parallel_size > max_dcp_size:\n                raise ValueError(\n                    \"`--decode-context-parallel-size` \"\n                    f\"({decode_context_parallel_size}) exceeds the maximum \"\n                    f\"supported value ({max_dcp_size}) for \"\n                    f\"`--tensor-parallel-size` ({tensor_parallel_size}) and \"\n                    f\"{total_num_kv_heads} model KV heads.\"\n                )\n\n            num_q_per_kv = total_num_attention_heads // total_num_kv_heads\n            if num_q_per_kv % decode_context_parallel_size != 0:\n                raise ValueError(\n                    \"The model's number of query heads per KV head \"\n                    f\"({num_q_per_kv}) must be divisible by \"\n                    \"`--decode-context-parallel-size` \"\n                    f\"({decode_context_parallel_size}) for GQA/MQA.\"\n                )\n\n        # torch_shm uses a single IPC queue to rank 0; DP>1 is\n        # incompatible because API servers can't know which\n        # CoreEngine the scheduler will assign work to. TP>1 is\n        # also not supported because this requires broadcasting\n        # MM tensors between all TP ranks.\n        if (\n            self.multimodal_config is not None\n            and self.multimodal_config.mm_tensor_ipc == \"torch_shm\"\n            and parallel_config.world_size_across_dp > 1\n        ):\n            raise ValueError(\n                \"mm_tensor_ipc='torch_shm' is not supported with \"","sourceCodeStart":1392,"sourceCodeEnd":1428,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/model.py#L1392-L1428","documentation":"With DCP on a GQA/MQA model, each KV head serves a group of query heads; that group must split evenly across the DCP ranks. Raised when (total_num_attention_heads // total_num_kv_heads) % decode_context_parallel_size != 0.","triggerScenarios":"Non-MLA model with DCP > 1 where the query-heads-per-KV-head ratio is not divisible by DCP — e.g. 48 query heads / 8 KV heads = 6 per group, DCP=4 fails (6 % 4 != 0).","commonSituations":"Choosing DCP as the largest remaining TP factor without checking the GQA group size; models with odd GQA ratios (6:1, 7:1) where only small DCP values divide cleanly.","solutions":["Pick a --decode-context-parallel-size that divides the query-heads-per-KV-head ratio exactly (valid values are divisors of num_attention_heads // num_key_value_heads).","Increase --tensor-parallel-size and choose a compatible DCP divisor.","Set --decode-context-parallel-size 1 to disable DCP."],"exampleFix":"# before (6 query heads per KV head)\nvllm serve model --tensor-parallel-size 16 --decode-context-parallel-size 4\n# after\nvllm serve model --tensor-parallel-size 16 --decode-context-parallel-size 3","handlingStrategy":"validation","validationCode":"def q_per_kv_divides_dcp(num_heads: int, kv_heads: int, dcp: int) -> bool:\n    return dcp <= 1 or (num_heads // kv_heads) % dcp == 0\n# validate before constructing ParallelConfig with DCP > 1","typeGuard":null,"tryCatchPattern":null,"preventionTips":["For GQA models, restrict DCP choices to divisors of num_attention_heads // num_key_value_heads.","Add a config lint step that checks all three DCP constraints (TP>KV, DCP<=TP//KV, ratio%DCP==0) together.","Watch out for odd GQA ratios (6:1, 7:1) which allow very few DCP values."],"tags":["parallelism","decode-context-parallel","gqa","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}