vllm-project/vllm · error · ValueError

TP sizes must be positive

Error message

TP sizes must be positive

What it means

get_moriio_remote_tp_rank maps a local TP rank to the corresponding remote (peer) TP rank in disaggregated prefill/decode. It raises ValueError('TP sizes must be positive') when local_tp_size or remote_tp_size is zero or negative, i.e. the peer's tensor-parallel size was never configured.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py:110

        IOEngineConfig,
    )

    logger.info("MoRIIO is available")
    MoRIIO_enabled = True
except ImportError:
    logger.error("MoRIIO is not available")
    MoRIIO_enabled = False


def is_moriio_available() -> bool:
    return MoRIIO_enabled


def get_moriio_remote_tp_rank(
    local_tp_rank: int, local_tp_size: int, remote_tp_size: int
) -> int:
    if local_tp_size <= 0 or remote_tp_size <= 0:
        raise ValueError("TP sizes must be positive")
    if local_tp_rank < 0 or local_tp_rank >= local_tp_size:
        raise ValueError(
            f"local_tp_rank {local_tp_rank} must be in [0, {local_tp_size})"
        )
    if remote_tp_size == local_tp_size:
        return local_tp_rank
    if remote_tp_size > local_tp_size:
        if remote_tp_size % local_tp_size != 0:
            raise ValueError(
                f"remote tp_size {remote_tp_size} must be a multiple of local "
                f"tp_size {local_tp_size} for heterogeneous-TP P/D"
            )
        return local_tp_rank * (remote_tp_size // local_tp_size)
    if local_tp_size % remote_tp_size != 0:
        raise ValueError(
            f"local tp_size {local_tp_size} must be a multiple of remote "
            f"tp_size {remote_tp_size} for heterogeneous-TP P/D"
        )

View on GitHub (pinned to c794754062)

Solutions

1. Verify both instances report a valid --tensor-parallel-size (>= 1)
2. Check where remote_tp_size is sourced (kv_transfer_params or handshake metadata) and that the peer populates it before transfers begin
3. Ensure startup ordering: peer metadata handshake completes before rank mapping is attempted

def validate_tp_sizes(*sizes: int) -> None:
    for s in sizes:
        if s <= 0:
            raise ValueError(f"TP size must be positive, got {s}")

def are_positive_tp_sizes(*sizes) -> bool:
    return all(isinstance(s, int) and s > 0 for s in sizes)

When it happens

Trigger: Calling get_moriio_remote_tp_rank with a tp size of 0 or a negative value; in practice the remote tp size is derived from kv_transfer_params/handshake metadata and defaults to 0 when the peer never announced it.

Common situations: Decode instance starts before the prefill registered its tp size; remote_tp_size config key misspelled or missing so it resolves to 0; multi-pod config lists mismatched dp/tp fields.

Related errors

• kv_connector_module_path cannot be an empty string.
• Invalid MoRIIO backend {backend!r} in kv_connector_extra_con
• Malformed zmq_address {zmq_address!r}: expected 'host:IP,han
• request_id {request_id!r} does not embed a peer zmq_address
• local_tp_rank {local_tp_rank} must be in [0, {local_tp_size}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/03b6d12f9555fad4. Report an issue: GitHub.