{"record":{"id":"cb4a913b864bd093","repo":"vllm-project/vllm","slug":"local-tp-size-local-tp-size-must-be-a-multiple-o","errorCode":null,"errorMessage":"local tp_size {local_tp_size} must be a multiple of remote tp_size {remote_tp_size} for heterogeneous-TP P/D","messagePattern":"local tp_size (.+?) must be a multiple of remote tp_size (.+?) for heterogeneous-TP P/D","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py","lineNumber":125,"sourceCode":"    local_tp_rank: int, local_tp_size: int, remote_tp_size: int\n) -> int:\n    if local_tp_size <= 0 or remote_tp_size <= 0:\n        raise ValueError(\"TP sizes must be positive\")\n    if local_tp_rank < 0 or local_tp_rank >= local_tp_size:\n        raise ValueError(\n            f\"local_tp_rank {local_tp_rank} must be in [0, {local_tp_size})\"\n        )\n    if remote_tp_size == local_tp_size:\n        return local_tp_rank\n    if remote_tp_size > local_tp_size:\n        if remote_tp_size % local_tp_size != 0:\n            raise ValueError(\n                f\"remote tp_size {remote_tp_size} must be a multiple of local \"\n                f\"tp_size {local_tp_size} for heterogeneous-TP P/D\"\n            )\n        return local_tp_rank * (remote_tp_size // local_tp_size)\n    if local_tp_size % remote_tp_size != 0:\n        raise ValueError(\n            f\"local tp_size {local_tp_size} must be a multiple of remote \"\n            f\"tp_size {remote_tp_size} for heterogeneous-TP P/D\"\n        )\n    return local_tp_rank // (local_tp_size // remote_tp_size)\n\n\ndef validate_moriio_heterogeneous_tp_kv_heads(\n    local_tp_size: int,\n    remote_tp_size: int,\n    total_num_kv_heads: int,\n    is_mla: bool,\n) -> None:\n    if is_mla or local_tp_size == remote_tp_size:\n        return\n    if local_tp_size <= 0 or remote_tp_size <= 0 or total_num_kv_heads <= 0:\n        raise ValueError(\"TP sizes and total_num_kv_heads must be positive\")\n    if min(local_tp_size, remote_tp_size) >= total_num_kv_heads:\n        return","sourceCodeStart":107,"sourceCodeEnd":143,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py#L107-L143","documentation":"Mirror case of the divisibility check in get_moriio_remote_tp_rank: when local_tp_size > remote_tp_size the mapping divides the local rank by the ratio, requiring local tp size to be a multiple of remote tp size. Non-divisible pairs (e.g. local=6, remote=4) raise this ValueError.","triggerScenarios":"Disaggregated P/D where the local instance has the larger TP count and it is not divisible by the remote one: (local=3, remote=2), (local=6, remote=4), etc.","commonSituations":"Prefill larger than decode with non-divisible sizes (TP 6 prefill / TP 4 decode); heterogeneous clusters assembled from mixed GPU counts.","solutions":["Make the larger TP size an exact multiple of the smaller (e.g. 4 and 8, 3 and 6, or equal sizes)","Prefer power-of-two TP sizes on both sides so divisibility holds automatically","Validate the pair at deployment time in the launcher/sidecar config"],"exampleFix":"# before: prefill TP=6, decode TP=4 -> raises\n# after: prefill TP=8, decode TP=4 -> ok","handlingStrategy":"validation","validationCode":"def validate_pd_tp_pair(local_tp: int, remote_tp: int) -> None:\n    lo, hi = min(local_tp, remote_tp), max(local_tp, remote_tp)\n    if hi % lo != 0:\n        raise ValueError(f\"heterogeneous-TP requires divisibility: {hi} % {lo} != 0\")","typeGuard":"def is_divisible_tp_pair(a: int, b: int) -> bool:\n    lo, hi = min(a, b), max(a, b)\n    return hi % lo == 0","tryCatchPattern":null,"preventionTips":["Keep P/D TP sizes equal or in exact multiple ratios (2:1, 4:1)","Document the topology contract wherever instance launch configs live"],"tags":["tensor-parallel","kv-transfer","heterogeneous-tp","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}