{"record":{"id":"23b4373e94335665","repo":"vllm-project/vllm","slug":"decode-context-parallelism-for-gqa-mqa-requires","errorCode":null,"errorMessage":"Decode context parallelism for GQA/MQA requires `--tensor-parallel-size` ({tensor_parallel_size}) to be greater than the model's total number of KV heads ({total_num_kv_heads}). Increase `--tensor-parallel-size` or set `--decode-context-parallel-size 1`.","messagePattern":"Decode context parallelism for GQA/MQA requires `--tensor-parallel-size` \\((.+?)\\) to be greater than the model's total number of KV heads \\((.+?)\\)\\. Increase `--tensor-parallel-size` or set `--decode-context-parallel-size 1`\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/model.py","lineNumber":1390,"sourceCode":"            )\n\n        if parallel_config.enable_expert_parallel:\n            self._verify_with_expert_parallelism()\n\n        pipeline_parallel_size = parallel_config.pipeline_parallel_size\n        if pipeline_parallel_size > 1 and not self.registry.is_pp_supported_model(\n            self.architectures, self\n        ):\n            raise NotImplementedError(\n                \"Pipeline parallelism is not supported for this model. \"\n                \"Supported models implement the `SupportsPP` interface.\"\n            )\n\n        decode_context_parallel_size = parallel_config.decode_context_parallel_size\n        if decode_context_parallel_size > 1 and not self.use_mla:\n            total_num_kv_heads = self.get_total_num_kv_heads()\n            if tensor_parallel_size <= total_num_kv_heads:\n                raise ValueError(\n                    \"Decode context parallelism for GQA/MQA requires \"\n                    f\"`--tensor-parallel-size` ({tensor_parallel_size}) to be \"\n                    \"greater than the model's total number of KV heads \"\n                    f\"({total_num_kv_heads}). Increase `--tensor-parallel-size` \"\n                    \"or set `--decode-context-parallel-size 1`.\"\n                )\n\n            max_dcp_size = tensor_parallel_size // total_num_kv_heads\n            if decode_context_parallel_size > max_dcp_size:\n                raise ValueError(\n                    \"`--decode-context-parallel-size` \"\n                    f\"({decode_context_parallel_size}) exceeds the maximum \"\n                    f\"supported value ({max_dcp_size}) for \"\n                    f\"`--tensor-parallel-size` ({tensor_parallel_size}) and \"\n                    f\"{total_num_kv_heads} model KV heads.\"\n                )\n\n            num_q_per_kv = total_num_attention_heads // total_num_kv_heads","sourceCodeStart":1372,"sourceCodeEnd":1408,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/model.py#L1372-L1408","documentation":"Decode context parallelism (DCP) shards KV heads across ranks for GQA/MQA models; it requires tensor_parallel_size to strictly exceed the model's total KV heads so there are ranks left over for context sharding. Raised when TP <= total_num_kv_heads on the non-MLA DCP path.","triggerScenarios":"verify_with_parallel_config runs with decode_context_parallel_size > 1, use_mla false, and tensor_parallel_size <= get_total_num_kv_heads() (e.g. 8 KV heads with TP=8 and DCP=2).","commonSituations":"Enabling --decode-context-parallel-size > 1 for long-context serving on a GQA model without raising TP first; assuming DCP works like an extra TP dimension with the same constraints.","solutions":["Increase --tensor-parallel-size so it is strictly greater than the model's num_key_value_heads.","Set --decode-context-parallel-size 1 to disable DCP.","If the model is MLA (e.g. DeepSeek), confirm the architecture is detected as MLA — the GQA path does not apply there."],"exampleFix":"# before (8 KV heads)\nvllm serve model --tensor-parallel-size 8 --decode-context-parallel-size 2\n# after\nvllm serve model --tensor-parallel-size 16 --decode-context-parallel-size 2","handlingStrategy":"validation","validationCode":"def dcp_gqa_ok(tp: int, dcp: int, total_kv_heads: int) -> bool:\n    return dcp <= 1 or tp > total_kv_heads\n# call with values from parallel_config and model_config.get_total_num_kv_heads()","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Before enabling DCP on GQA models, verify TP strictly exceeds num_key_value_heads.","Compute the DCP budget (TP // KV heads) as part of launch-flag planning.","Keep DCP=1 as the default; enable it only after arithmetic validation passes."],"tags":["parallelism","decode-context-parallel","gqa","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}