vllm-project/vllm · error · ValueError

local_tp_rank {local_tp_rank} must be in [0, {local_tp_size}

Error message

local_tp_rank {local_tp_rank} must be in [0, {local_tp_size})

What it means

get_moriio_remote_tp_rank validates that local_tp_rank lies in [0, local_tp_size). The error means the rank passed in is outside the tensor-parallel group range, e.g. a global/DP-rank was supplied where a TP rank was expected, or the tp size used for validation is smaller than the real one.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py:112

    logger.info("MoRIIO is available")
    MoRIIO_enabled = True
except ImportError:
    logger.error("MoRIIO is not available")
    MoRIIO_enabled = False


def is_moriio_available() -> bool:
    return MoRIIO_enabled


def get_moriio_remote_tp_rank(
    local_tp_rank: int, local_tp_size: int, remote_tp_size: int
) -> int:
    if local_tp_size <= 0 or remote_tp_size <= 0:
        raise ValueError("TP sizes must be positive")
    if local_tp_rank < 0 or local_tp_rank >= local_tp_size:
        raise ValueError(
            f"local_tp_rank {local_tp_rank} must be in [0, {local_tp_size})"
        )
    if remote_tp_size == local_tp_size:
        return local_tp_rank
    if remote_tp_size > local_tp_size:
        if remote_tp_size % local_tp_size != 0:
            raise ValueError(
                f"remote tp_size {remote_tp_size} must be a multiple of local "
                f"tp_size {local_tp_size} for heterogeneous-TP P/D"
            )
        return local_tp_rank * (remote_tp_size // local_tp_size)
    if local_tp_size % remote_tp_size != 0:
        raise ValueError(
            f"local tp_size {local_tp_size} must be a multiple of remote "
            f"tp_size {remote_tp_size} for heterogeneous-TP P/D"
        )
    return local_tp_rank // (local_tp_size // remote_tp_size)

View on GitHub (pinned to c794754062)

Solutions

1. Pass the true TP rank (e.g. vllm_config.parallel_config.tensor_parallel_rank / tp_group.rank_in_group), not world or DP rank
2. Confirm local_tp_size matches the instance's --tensor-parallel-size
3. Add an assert/range check at the call site before invoking the mapper

Example fix

// before
remote_rank = get_moriio_remote_tp_rank(dp_rank, tp_size, remote_tp_size)

// after
remote_rank = get_moriio_remote_tp_rank(tp_rank, tp_size, remote_tp_size)

assert 0 <= tp_rank < tp_size, f"tp_rank {tp_rank} out of range for tp_size {tp_size}"

def is_valid_tp_rank(rank, tp_size) -> bool:
    return isinstance(rank, int) and isinstance(tp_size, int) and 0 <= rank < tp_size

When it happens

Trigger: Passing get_tensor_model_parallel_rank vs world rank confusion; calling with local_tp_rank == local_tp_size (off-by-one from a range end); mixing remote tp size into the local validation slot.

Common situations: Refactors that change which rank variable is threaded into the connector; running with DP>1 and accidentally using the data-parallel rank; unit tests calling the helper with arbitrary ranks.

Related errors

• TP sizes must be positive
• TP sizes and total_num_kv_heads must be positive
• HTTP request failed: {0}
• kv_connector_module_path cannot be an empty string.
• Hf3fsClient.check Failed

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/69e556c009dbc0cb. Report an issue: GitHub.