{"record":{"id":"8a94eee2b843fde3","repo":"vllm-project/vllm","slug":"tp-sizes-and-total-num-kv-heads-must-be-positive","errorCode":null,"errorMessage":"TP sizes and total_num_kv_heads must be positive","messagePattern":"TP sizes and total_num_kv_heads must be positive","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py","lineNumber":141,"sourceCode":"        return local_tp_rank * (remote_tp_size // local_tp_size)\n    if local_tp_size % remote_tp_size != 0:\n        raise ValueError(\n            f\"local tp_size {local_tp_size} must be a multiple of remote \"\n            f\"tp_size {remote_tp_size} for heterogeneous-TP P/D\"\n        )\n    return local_tp_rank // (local_tp_size // remote_tp_size)\n\n\ndef validate_moriio_heterogeneous_tp_kv_heads(\n    local_tp_size: int,\n    remote_tp_size: int,\n    total_num_kv_heads: int,\n    is_mla: bool,\n) -> None:\n    if is_mla or local_tp_size == remote_tp_size:\n        return\n    if local_tp_size <= 0 or remote_tp_size <= 0 or total_num_kv_heads <= 0:\n        raise ValueError(\"TP sizes and total_num_kv_heads must be positive\")\n    if min(local_tp_size, remote_tp_size) >= total_num_kv_heads:\n        return\n    raise NotImplementedError(\n        \"MoRIIO heterogeneous TP requires replicated KV heads on both \"\n        f\"prefill and decode. Got total_num_kv_heads={total_num_kv_heads}, \"\n        f\"local_tp_size={local_tp_size}, remote_tp_size={remote_tp_size}.\"\n    )\n\n\ndef get_moriio_expected_ack_count(producer_tp_size: int, consumer_tp_size: int) -> int:\n    if producer_tp_size <= 0 or consumer_tp_size <= 0:\n        raise ValueError(\"TP sizes must be positive\")\n    if consumer_tp_size <= producer_tp_size:\n        return 1\n    if consumer_tp_size % producer_tp_size != 0:\n        raise ValueError(\n            f\"consumer tp_size {consumer_tp_size} must be a multiple of \"\n            f\"producer tp_size {producer_tp_size} for heterogeneous-TP P/D\"","sourceCodeStart":123,"sourceCodeEnd":159,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py#L123-L159","documentation":"validate_moriio_heterogeneous_tp_kv_heads checks KV-head replication constraints for heterogeneous-TP P/D. Before the main check it asserts local_tp_size, remote_tp_size, and total_num_kv_heads are all positive; zero/negative values (unset config) raise this ValueError.","triggerScenarios":"Calling the validator with a tp size or total_num_kv_heads of 0/negative, e.g. model config not loaded so num_key_value_heads resolves to 0, or a remote tp size defaulting to 0 before handshake.","commonSituations":"Initializing the connector before model config is fully populated; misparsed model config where attention KV heads read as 0; test harnesses calling validators with placeholder values.","solutions":["Confirm the model config exposes a positive num_key_value_heads (and derived total_num_kv_heads)","Ensure both local and remote tp sizes are configured positive before validation runs","Log the three values at connector init to catch unset fields early"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if local_tp_size <= 0 or remote_tp_size <= 0 or total_num_kv_heads <= 0:\n    raise ValueError(f\"invalid geometry: tp=({local_tp_size},{remote_tp_size}) kv_heads={total_num_kv_heads}\")","typeGuard":"def has_positive_geometry(tp_a, tp_b, kv_heads) -> bool:\n    return all(isinstance(v, int) and v > 0 for v in (tp_a, tp_b, kv_heads))","tryCatchPattern":null,"preventionTips":["Load and assert model config (num_key_value_heads) before connector init","Fail startup on zero-valued parallel or head fields instead of deferring"],"tags":["tensor-parallel","kv-transfer","model-config","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}