vllm-project/vllm · error · ValueError

The model's number of query heads per KV head ({num_q_per_kv

Error message

The model's number of query heads per KV head ({num_q_per_kv}) must be divisible by `--decode-context-parallel-size` ({decode_context_parallel_size}) for GQA/MQA.

What it means

With DCP on a GQA/MQA model, each KV head serves a group of query heads; that group must split evenly across the DCP ranks. Raised when (total_num_attention_heads // total_num_kv_heads) % decode_context_parallel_size != 0.

Source

Thrown at vllm/config/model.py:1410

                    f"`--tensor-parallel-size` ({tensor_parallel_size}) to be "
                    "greater than the model's total number of KV heads "
                    f"({total_num_kv_heads}). Increase `--tensor-parallel-size` "
                    "or set `--decode-context-parallel-size 1`."
                )

            max_dcp_size = tensor_parallel_size // total_num_kv_heads
            if decode_context_parallel_size > max_dcp_size:
                raise ValueError(
                    "`--decode-context-parallel-size` "
                    f"({decode_context_parallel_size}) exceeds the maximum "
                    f"supported value ({max_dcp_size}) for "
                    f"`--tensor-parallel-size` ({tensor_parallel_size}) and "
                    f"{total_num_kv_heads} model KV heads."
                )

            num_q_per_kv = total_num_attention_heads // total_num_kv_heads
            if num_q_per_kv % decode_context_parallel_size != 0:
                raise ValueError(
                    "The model's number of query heads per KV head "
                    f"({num_q_per_kv}) must be divisible by "
                    "`--decode-context-parallel-size` "
                    f"({decode_context_parallel_size}) for GQA/MQA."
                )

        # torch_shm uses a single IPC queue to rank 0; DP>1 is
        # incompatible because API servers can't know which
        # CoreEngine the scheduler will assign work to. TP>1 is
        # also not supported because this requires broadcasting
        # MM tensors between all TP ranks.
        if (
            self.multimodal_config is not None
            and self.multimodal_config.mm_tensor_ipc == "torch_shm"
            and parallel_config.world_size_across_dp > 1
        ):
            raise ValueError(
                "mm_tensor_ipc='torch_shm' is not supported with "

View on GitHub (pinned to c794754062)

Solutions

Pick a --decode-context-parallel-size that divides the query-heads-per-KV-head ratio exactly (valid values are divisors of num_attention_heads // num_key_value_heads).
Increase --tensor-parallel-size and choose a compatible DCP divisor.
Set --decode-context-parallel-size 1 to disable DCP.

Example fix

# before (6 query heads per KV head)
vllm serve model --tensor-parallel-size 16 --decode-context-parallel-size 4
# after
vllm serve model --tensor-parallel-size 16 --decode-context-parallel-size 3

Defensive patterns

Strategy: validation

Validate before calling

def q_per_kv_divides_dcp(num_heads: int, kv_heads: int, dcp: int) -> bool:
    return dcp <= 1 or (num_heads // kv_heads) % dcp == 0
# validate before constructing ParallelConfig with DCP > 1

Prevention

For GQA models, restrict DCP choices to divisors of num_attention_heads // num_key_value_heads.
Add a config lint step that checks all three DCP constraints (TP>KV, DCP<=TP//KV, ratio%DCP==0) together.
Watch out for odd GQA ratios (6:1, 7:1) which allow very few DCP values.

When it happens

Trigger: Non-MLA model with DCP > 1 where the query-heads-per-KV-head ratio is not divisible by DCP — e.g. 48 query heads / 8 KV heads = 6 per group, DCP=4 fails (6 % 4 != 0).

Common situations: Choosing DCP as the largest remaining TP factor without checking the GQA group size; models with odd GQA ratios (6:1, 7:1) where only small DCP values divide cleanly.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/d97d5d8528e2b364. Report an issue: GitHub.