What causes "tp_size={tp} must be divisible by dcp_size={dcp}." in vllm-project/vllm?

Pick a DCP size that divides TP, e.g. TP=4 supports DCP in {1, 2, 4}.

vllm-project/vllm · error · ValueError

tp_size={tp} must be divisible by dcp_size={dcp}.

Error message

tp_size={tp} must be divisible by dcp_size={dcp}.

What it means

When PCP is disabled, decode context parallelism (DCP) reuses the tensor-parallel ranks, so the TP size must be divisible by the DCP size. ParallelConfig raises tp % dcp != 0 in this branch.

Source

Thrown at vllm/config/parallel.py:532

                )
        else:
            if self.eplb_config.num_redundant_experts != 0:
                raise ValueError(
                    "num_redundant_experts is set to "
                    f"{self.eplb_config.num_redundant_experts} but EPLB is not "
                    "enabled. Either enable EPLB or unset "
                    "num_redundant_experts."
                )

        tp = self.tensor_parallel_size
        pcp = self.prefill_context_parallel_size
        dcp = self.decode_context_parallel_size
        if pcp > 1 and self.data_parallel_size > 1:
            raise ValueError("PCP does not support data parallelism yet.")
        if pcp == 1:
            # DCP reuses the TP ranks when PCP is disabled.
            if tp % dcp != 0:
                raise ValueError(f"tp_size={tp} must be divisible by dcp_size={dcp}.")
        elif dcp not in (1, pcp, tp * pcp):
            raise ValueError(
                "When PCP is enabled, DCP must be disabled, span the PCP "
                "axis, or span the full TP x PCP axis. "
                f"Got TP={tp}, PCP={pcp}, DCP={dcp}; valid DCP sizes are "
                f"{sorted({1, pcp, tp * pcp})}."
            )

        if self.dcp_comm_backend == "a2a" and self.decode_context_parallel_size <= 1:
            raise ValueError(
                "dcp_comm_backend='a2a' requires decode_context_parallel_size > 1."
            )

        return self

    @property
    def world_size_across_dp(self) -> int:
        """Process world size across TP, PCP, PP, and DP."""

View on GitHub (pinned to c794754062)

Solutions

Pick a DCP size that divides TP, e.g. TP=4 supports DCP in {1, 2, 4}.
Or raise TP to a multiple of the desired DCP size.
Set DCP back to 1 if decode context parallelism is not needed.

Example fix

# before
--tensor-parallel-size 4 --decode-context-parallel-size 3
# after
--tensor-parallel-size 4 --decode-context-parallel-size 2

Defensive patterns

Strategy: validation

Validate before calling

def tp_dcp_valid(tp: int, dcp: int, pcp: int = 1) -> bool:
    return pcp > 1 or tp % dcp == 0

assert tp_dcp_valid(4, 2)

Prevention

Compute DCP as a divisor of TP (tp // k) in launch scripts instead of a hardcoded constant.
When changing TP, re-derive every dependent parallel size in the same change.

When it happens

Trigger: Config with prefill_context_parallel_size=1 and a decode_context_parallel_size that does not divide tensor_parallel_size, e.g. TP=4 with DCP=3, or TP=2 with DCP=4.

Common situations: Hand-tuning DCP for long-context decode without checking rank factorization; changing TP size after DCP was already set.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/8ce9b4cdaf700441. Report an issue: GitHub.