vllm-project/vllm · error · ValueError

TP sizes and total_num_kv_heads must be positive

Error message

TP sizes and total_num_kv_heads must be positive

What it means

validate_moriio_heterogeneous_tp_kv_heads checks KV-head replication constraints for heterogeneous-TP P/D. Before the main check it asserts local_tp_size, remote_tp_size, and total_num_kv_heads are all positive; zero/negative values (unset config) raise this ValueError.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py:141

        return local_tp_rank * (remote_tp_size // local_tp_size)
    if local_tp_size % remote_tp_size != 0:
        raise ValueError(
            f"local tp_size {local_tp_size} must be a multiple of remote "
            f"tp_size {remote_tp_size} for heterogeneous-TP P/D"
        )
    return local_tp_rank // (local_tp_size // remote_tp_size)


def validate_moriio_heterogeneous_tp_kv_heads(
    local_tp_size: int,
    remote_tp_size: int,
    total_num_kv_heads: int,
    is_mla: bool,
) -> None:
    if is_mla or local_tp_size == remote_tp_size:
        return
    if local_tp_size <= 0 or remote_tp_size <= 0 or total_num_kv_heads <= 0:
        raise ValueError("TP sizes and total_num_kv_heads must be positive")
    if min(local_tp_size, remote_tp_size) >= total_num_kv_heads:
        return
    raise NotImplementedError(
        "MoRIIO heterogeneous TP requires replicated KV heads on both "
        f"prefill and decode. Got total_num_kv_heads={total_num_kv_heads}, "
        f"local_tp_size={local_tp_size}, remote_tp_size={remote_tp_size}."
    )


def get_moriio_expected_ack_count(producer_tp_size: int, consumer_tp_size: int) -> int:
    if producer_tp_size <= 0 or consumer_tp_size <= 0:
        raise ValueError("TP sizes must be positive")
    if consumer_tp_size <= producer_tp_size:
        return 1
    if consumer_tp_size % producer_tp_size != 0:
        raise ValueError(
            f"consumer tp_size {consumer_tp_size} must be a multiple of "
            f"producer tp_size {producer_tp_size} for heterogeneous-TP P/D"

View on GitHub (pinned to c794754062)

Solutions

1. Confirm the model config exposes a positive num_key_value_heads (and derived total_num_kv_heads)
2. Ensure both local and remote tp sizes are configured positive before validation runs
3. Log the three values at connector init to catch unset fields early

if local_tp_size <= 0 or remote_tp_size <= 0 or total_num_kv_heads <= 0:
    raise ValueError(f"invalid geometry: tp=({local_tp_size},{remote_tp_size}) kv_heads={total_num_kv_heads}")

def has_positive_geometry(tp_a, tp_b, kv_heads) -> bool:
    return all(isinstance(v, int) and v > 0 for v in (tp_a, tp_b, kv_heads))

When it happens

Trigger: Calling the validator with a tp size or total_num_kv_heads of 0/negative, e.g. model config not loaded so num_key_value_heads resolves to 0, or a remote tp size defaulting to 0 before handshake.

Common situations: Initializing the connector before model config is fully populated; misparsed model config where attention KV heads read as 0; test harnesses calling validators with placeholder values.

Related errors

• TP sizes must be positive
• local_tp_rank {local_tp_rank} must be in [0, {local_tp_size}
• MoRIIO heterogeneous TP requires replicated KV heads on both
• HTTP request failed: {0}
• kv_connector_module_path cannot be an empty string.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/8a94eee2b843fde3. Report an issue: GitHub.