vllm-project/vllm · error · ValueError

consumer tp_size {consumer_tp_size} must be a multiple of pr

Error message

consumer tp_size {consumer_tp_size} must be a multiple of producer tp_size {producer_tp_size} for heterogeneous-TP P/D

What it means

get_moriio_expected_ack_count requires that when consumer_tp_size > producer_tp_size, the consumer size is an exact multiple of the producer size (the ack count is the ratio consumer/producer). Non-divisible sizes like producer=4 / consumer=6 raise this ValueError.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py:157

        return
    if local_tp_size <= 0 or remote_tp_size <= 0 or total_num_kv_heads <= 0:
        raise ValueError("TP sizes and total_num_kv_heads must be positive")
    if min(local_tp_size, remote_tp_size) >= total_num_kv_heads:
        return
    raise NotImplementedError(
        "MoRIIO heterogeneous TP requires replicated KV heads on both "
        f"prefill and decode. Got total_num_kv_heads={total_num_kv_heads}, "
        f"local_tp_size={local_tp_size}, remote_tp_size={remote_tp_size}."
    )


def get_moriio_expected_ack_count(producer_tp_size: int, consumer_tp_size: int) -> int:
    if producer_tp_size <= 0 or consumer_tp_size <= 0:
        raise ValueError("TP sizes must be positive")
    if consumer_tp_size <= producer_tp_size:
        return 1
    if consumer_tp_size % producer_tp_size != 0:
        raise ValueError(
            f"consumer tp_size {consumer_tp_size} must be a multiple of "
            f"producer tp_size {producer_tp_size} for heterogeneous-TP P/D"
        )
    return consumer_tp_size // producer_tp_size


def resolve_moriio_transfer_ack(
    ack: MoRIIOTransferAck | TransferId,
    producer_tp_size: int,
    live_transfer_ids: Collection[TransferId],
    notification_counts: dict[TransferId, int],
    completed_transfer_ids: set[TransferId],
) -> TransferId | None:
    if isinstance(ack, str):
        ack = MoRIIOTransferAck(ack)
    transfer_id = ack.transfer_id
    if transfer_id not in live_transfer_ids:
        return None

View on GitHub (pinned to c794754062)

Solutions

Choose P/D TP sizes where decode TP is a multiple of prefill TP (4->8, 3->6) or both equal
Standardize on power-of-two TP sizes across the cluster
Assert divisibility in the deployment script before bringing up instances

Example fix

# before: prefill TP=4, decode TP=6 -> raises
# after: prefill TP=3, decode TP=6 -> ok (ack count 2)

Defensive patterns

Strategy: validation

Validate before calling

if consumer_tp_size > producer_tp_size:
    assert consumer_tp_size % producer_tp_size == 0, (
        f"consumer TP {consumer_tp_size} must be a multiple of producer TP {producer_tp_size}"
    )

Type guard

def is_ack_divisible_tp_pair(producer_tp: int, consumer_tp: int) -> bool:
    return consumer_tp <= producer_tp or consumer_tp % producer_tp == 0

Prevention

Validate the P/D TP ratio once at cluster bring-up
Use shared topology config for both roles so they cannot drift

When it happens

Trigger: Disaggregated P/D with decode (consumer) TP larger than prefill (producer) TP and not divisible by it: producer=2/consumer=3, producer=4/consumer=6, etc.

Common situations: Scaling out decode replicas with arbitrary GPU counts; combining with the rank-mapping divisibility rules (546/547) but forgetting the ack path has the same constraint.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/6cb1723457d10b4a. Report an issue: GitHub.