vllm-project/vllm · error · ValueError

remote tp_size {remote_tp_size} must be a multiple of local

Error message

remote tp_size {remote_tp_size} must be a multiple of local tp_size {local_tp_size} for heterogeneous-TP P/D

What it means

In get_moriio_remote_tp_rank, when remote_tp_size > local_tp_size the mapping multiplies the local rank by the size ratio, which requires remote tp size to be an integer multiple of local tp size. Non-divisible heterogeneous TP (e.g. prefill TP=4, decode TP=6) raises this ValueError.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py:119

def is_moriio_available() -> bool:
    return MoRIIO_enabled


def get_moriio_remote_tp_rank(
    local_tp_rank: int, local_tp_size: int, remote_tp_size: int
) -> int:
    if local_tp_size <= 0 or remote_tp_size <= 0:
        raise ValueError("TP sizes must be positive")
    if local_tp_rank < 0 or local_tp_rank >= local_tp_size:
        raise ValueError(
            f"local_tp_rank {local_tp_rank} must be in [0, {local_tp_size})"
        )
    if remote_tp_size == local_tp_size:
        return local_tp_rank
    if remote_tp_size > local_tp_size:
        if remote_tp_size % local_tp_size != 0:
            raise ValueError(
                f"remote tp_size {remote_tp_size} must be a multiple of local "
                f"tp_size {local_tp_size} for heterogeneous-TP P/D"
            )
        return local_tp_rank * (remote_tp_size // local_tp_size)
    if local_tp_size % remote_tp_size != 0:
        raise ValueError(
            f"local tp_size {local_tp_size} must be a multiple of remote "
            f"tp_size {remote_tp_size} for heterogeneous-TP P/D"
        )
    return local_tp_rank // (local_tp_size // remote_tp_size)


def validate_moriio_heterogeneous_tp_kv_heads(
    local_tp_size: int,
    remote_tp_size: int,
    total_num_kv_heads: int,
    is_mla: bool,
) -> None:

View on GitHub (pinned to c794754062)

Solutions

1. Pick TP sizes where one divides the other exactly, e.g. prefill TP=4 / decode TP=8 or both equal
2. If asymmetric scaling is required, scale by powers of two (2:1, 4:1) which always satisfy divisibility
3. Add a startup config check on both instances so the mismatch surfaces before any request is routed

Example fix

# before
python -m vllm.entrypoints.openai.api_server --role prefill  --tensor-parallel-size 4
python -m vllm.entrypoints.openai.api_server --role decode   --tensor-parallel-size 6

# after
python -m vllm.entrypoints.openai.api_server --role prefill  --tensor-parallel-size 3
python -m vllm.entrypoints.openai.api_server --role decode   --tensor-parallel-size 6

def validate_pd_tp_pair(local_tp: int, remote_tp: int) -> None:
    lo, hi = min(local_tp, remote_tp), max(local_tp, remote_tp)
    if hi % lo != 0:
        raise ValueError(f"heterogeneous-TP requires divisibility: {hi} % {lo} != 0")

def is_divisible_tp_pair(a: int, b: int) -> bool:
    lo, hi = min(a, b), max(a, b)
    return hi % lo == 0

When it happens

Trigger: Disaggregated P/D with tensor-parallel sizes where the larger is not a multiple of the smaller: (local=2, remote=3), (local=4, remote=6), etc.

Common situations: Sizing decode GPUs differently from prefill GPUs (e.g. 8-way prefill, 12-way decode) without keeping divisibility; incremental cluster changes that break an earlier 2:1 ratio.

Related errors

• local tp_size {local_tp_size} must be a multiple of remote t
• consumer tp_size {consumer_tp_size} must be a multiple of pr
• TP sizes must be positive
• MoRIIO heterogeneous TP requires replicated KV heads on both
• --use-replayssm is incompatible with KV connectors (P/D disa

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/6e2af468212d0a86. Report an issue: GitHub.