vllm-project/vllm · error · ValueError

Decode context parallelism for GQA/MQA requires `--tensor-pa

Error message

Decode context parallelism for GQA/MQA requires `--tensor-parallel-size` ({tensor_parallel_size}) to be greater than the model's total number of KV heads ({total_num_kv_heads}). Increase `--tensor-parallel-size` or set `--decode-context-parallel-size 1`.

What it means

Decode context parallelism (DCP) shards KV heads across ranks for GQA/MQA models; it requires tensor_parallel_size to strictly exceed the model's total KV heads so there are ranks left over for context sharding. Raised when TP <= total_num_kv_heads on the non-MLA DCP path.

Source

Thrown at vllm/config/model.py:1390

            )

        if parallel_config.enable_expert_parallel:
            self._verify_with_expert_parallelism()

        pipeline_parallel_size = parallel_config.pipeline_parallel_size
        if pipeline_parallel_size > 1 and not self.registry.is_pp_supported_model(
            self.architectures, self
        ):
            raise NotImplementedError(
                "Pipeline parallelism is not supported for this model. "
                "Supported models implement the `SupportsPP` interface."
            )

        decode_context_parallel_size = parallel_config.decode_context_parallel_size
        if decode_context_parallel_size > 1 and not self.use_mla:
            total_num_kv_heads = self.get_total_num_kv_heads()
            if tensor_parallel_size <= total_num_kv_heads:
                raise ValueError(
                    "Decode context parallelism for GQA/MQA requires "
                    f"`--tensor-parallel-size` ({tensor_parallel_size}) to be "
                    "greater than the model's total number of KV heads "
                    f"({total_num_kv_heads}). Increase `--tensor-parallel-size` "
                    "or set `--decode-context-parallel-size 1`."
                )

            max_dcp_size = tensor_parallel_size // total_num_kv_heads
            if decode_context_parallel_size > max_dcp_size:
                raise ValueError(
                    "`--decode-context-parallel-size` "
                    f"({decode_context_parallel_size}) exceeds the maximum "
                    f"supported value ({max_dcp_size}) for "
                    f"`--tensor-parallel-size` ({tensor_parallel_size}) and "
                    f"{total_num_kv_heads} model KV heads."
                )

            num_q_per_kv = total_num_attention_heads // total_num_kv_heads

View on GitHub (pinned to c794754062)

Solutions

1. Increase --tensor-parallel-size so it is strictly greater than the model's num_key_value_heads.
2. Set --decode-context-parallel-size 1 to disable DCP.
3. If the model is MLA (e.g. DeepSeek), confirm the architecture is detected as MLA — the GQA path does not apply there.

Example fix

# before (8 KV heads)
vllm serve model --tensor-parallel-size 8 --decode-context-parallel-size 2
# after
vllm serve model --tensor-parallel-size 16 --decode-context-parallel-size 2

def dcp_gqa_ok(tp: int, dcp: int, total_kv_heads: int) -> bool:
    return dcp <= 1 or tp > total_kv_heads
# call with values from parallel_config and model_config.get_total_num_kv_heads()

When it happens

Trigger: verify_with_parallel_config runs with decode_context_parallel_size > 1, use_mla false, and tensor_parallel_size <= get_total_num_kv_heads() (e.g. 8 KV heads with TP=8 and DCP=2).

Common situations: Enabling --decode-context-parallel-size > 1 for long-context serving on a GQA model without raising TP first; assuming DCP works like an extra TP dimension with the same constraints.

Related errors

• `--decode-context-parallel-size` ({decode_context_parallel_s
• The model's number of query heads per KV head ({num_q_per_kv
• Number of experts in the model must be greater than 0 when e
• mm_tensor_ipc='torch_shm' is not supported with data_paralle
• {kind} parsing is not available for model `{model_id}`

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/23b4373e94335665. Report an issue: GitHub.