vllm-project/vllm · error · ValueError

local tp_size {local_tp_size} must be a multiple of remote t

Error message

local tp_size {local_tp_size} must be a multiple of remote tp_size {remote_tp_size} for heterogeneous-TP P/D

What it means

Mirror case of the divisibility check in get_moriio_remote_tp_rank: when local_tp_size > remote_tp_size the mapping divides the local rank by the ratio, requiring local tp size to be a multiple of remote tp size. Non-divisible pairs (e.g. local=6, remote=4) raise this ValueError.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py:125

    local_tp_rank: int, local_tp_size: int, remote_tp_size: int
) -> int:
    if local_tp_size <= 0 or remote_tp_size <= 0:
        raise ValueError("TP sizes must be positive")
    if local_tp_rank < 0 or local_tp_rank >= local_tp_size:
        raise ValueError(
            f"local_tp_rank {local_tp_rank} must be in [0, {local_tp_size})"
        )
    if remote_tp_size == local_tp_size:
        return local_tp_rank
    if remote_tp_size > local_tp_size:
        if remote_tp_size % local_tp_size != 0:
            raise ValueError(
                f"remote tp_size {remote_tp_size} must be a multiple of local "
                f"tp_size {local_tp_size} for heterogeneous-TP P/D"
            )
        return local_tp_rank * (remote_tp_size // local_tp_size)
    if local_tp_size % remote_tp_size != 0:
        raise ValueError(
            f"local tp_size {local_tp_size} must be a multiple of remote "
            f"tp_size {remote_tp_size} for heterogeneous-TP P/D"
        )
    return local_tp_rank // (local_tp_size // remote_tp_size)


def validate_moriio_heterogeneous_tp_kv_heads(
    local_tp_size: int,
    remote_tp_size: int,
    total_num_kv_heads: int,
    is_mla: bool,
) -> None:
    if is_mla or local_tp_size == remote_tp_size:
        return
    if local_tp_size <= 0 or remote_tp_size <= 0 or total_num_kv_heads <= 0:
        raise ValueError("TP sizes and total_num_kv_heads must be positive")
    if min(local_tp_size, remote_tp_size) >= total_num_kv_heads:
        return

View on GitHub (pinned to c794754062)

Solutions

1. Make the larger TP size an exact multiple of the smaller (e.g. 4 and 8, 3 and 6, or equal sizes)
2. Prefer power-of-two TP sizes on both sides so divisibility holds automatically
3. Validate the pair at deployment time in the launcher/sidecar config

Example fix

# before: prefill TP=6, decode TP=4 -> raises
# after: prefill TP=8, decode TP=4 -> ok

def validate_pd_tp_pair(local_tp: int, remote_tp: int) -> None:
    lo, hi = min(local_tp, remote_tp), max(local_tp, remote_tp)
    if hi % lo != 0:
        raise ValueError(f"heterogeneous-TP requires divisibility: {hi} % {lo} != 0")

def is_divisible_tp_pair(a: int, b: int) -> bool:
    lo, hi = min(a, b), max(a, b)
    return hi % lo == 0

When it happens

Trigger: Disaggregated P/D where the local instance has the larger TP count and it is not divisible by the remote one: (local=3, remote=2), (local=6, remote=4), etc.

Common situations: Prefill larger than decode with non-divisible sizes (TP 6 prefill / TP 4 decode); heterogeneous clusters assembled from mixed GPU counts.

Related errors

• remote tp_size {remote_tp_size} must be a multiple of local
• consumer tp_size {consumer_tp_size} must be a multiple of pr
• TP sizes must be positive
• MoRIIO heterogeneous TP requires replicated KV heads on both
• --use-replayssm is incompatible with KV connectors (P/D disa

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/cb4a913b864bd093. Report an issue: GitHub.