{"record":{"id":"6e2af468212d0a86","repo":"vllm-project/vllm","slug":"remote-tp-size-remote-tp-size-must-be-a-multiple","errorCode":null,"errorMessage":"remote tp_size {remote_tp_size} must be a multiple of local tp_size {local_tp_size} for heterogeneous-TP P/D","messagePattern":"remote tp_size (.+?) must be a multiple of local tp_size (.+?) for heterogeneous-TP P/D","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py","lineNumber":119,"sourceCode":"\ndef is_moriio_available() -> bool:\n    return MoRIIO_enabled\n\n\ndef get_moriio_remote_tp_rank(\n    local_tp_rank: int, local_tp_size: int, remote_tp_size: int\n) -> int:\n    if local_tp_size <= 0 or remote_tp_size <= 0:\n        raise ValueError(\"TP sizes must be positive\")\n    if local_tp_rank < 0 or local_tp_rank >= local_tp_size:\n        raise ValueError(\n            f\"local_tp_rank {local_tp_rank} must be in [0, {local_tp_size})\"\n        )\n    if remote_tp_size == local_tp_size:\n        return local_tp_rank\n    if remote_tp_size > local_tp_size:\n        if remote_tp_size % local_tp_size != 0:\n            raise ValueError(\n                f\"remote tp_size {remote_tp_size} must be a multiple of local \"\n                f\"tp_size {local_tp_size} for heterogeneous-TP P/D\"\n            )\n        return local_tp_rank * (remote_tp_size // local_tp_size)\n    if local_tp_size % remote_tp_size != 0:\n        raise ValueError(\n            f\"local tp_size {local_tp_size} must be a multiple of remote \"\n            f\"tp_size {remote_tp_size} for heterogeneous-TP P/D\"\n        )\n    return local_tp_rank // (local_tp_size // remote_tp_size)\n\n\ndef validate_moriio_heterogeneous_tp_kv_heads(\n    local_tp_size: int,\n    remote_tp_size: int,\n    total_num_kv_heads: int,\n    is_mla: bool,\n) -> None:","sourceCodeStart":101,"sourceCodeEnd":137,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py#L101-L137","documentation":"In get_moriio_remote_tp_rank, when remote_tp_size > local_tp_size the mapping multiplies the local rank by the size ratio, which requires remote tp size to be an integer multiple of local tp size. Non-divisible heterogeneous TP (e.g. prefill TP=4, decode TP=6) raises this ValueError.","triggerScenarios":"Disaggregated P/D with tensor-parallel sizes where the larger is not a multiple of the smaller: (local=2, remote=3), (local=4, remote=6), etc.","commonSituations":"Sizing decode GPUs differently from prefill GPUs (e.g. 8-way prefill, 12-way decode) without keeping divisibility; incremental cluster changes that break an earlier 2:1 ratio.","solutions":["Pick TP sizes where one divides the other exactly, e.g. prefill TP=4 / decode TP=8 or both equal","If asymmetric scaling is required, scale by powers of two (2:1, 4:1) which always satisfy divisibility","Add a startup config check on both instances so the mismatch surfaces before any request is routed"],"exampleFix":"# before\npython -m vllm.entrypoints.openai.api_server --role prefill  --tensor-parallel-size 4\npython -m vllm.entrypoints.openai.api_server --role decode   --tensor-parallel-size 6\n\n# after\npython -m vllm.entrypoints.openai.api_server --role prefill  --tensor-parallel-size 3\npython -m vllm.entrypoints.openai.api_server --role decode   --tensor-parallel-size 6","handlingStrategy":"validation","validationCode":"def validate_pd_tp_pair(local_tp: int, remote_tp: int) -> None:\n    lo, hi = min(local_tp, remote_tp), max(local_tp, remote_tp)\n    if hi % lo != 0:\n        raise ValueError(f\"heterogeneous-TP requires divisibility: {hi} % {lo} != 0\")","typeGuard":"def is_divisible_tp_pair(a: int, b: int) -> bool:\n    lo, hi = min(a, b), max(a, b)\n    return hi % lo == 0","tryCatchPattern":null,"preventionTips":["Assert P/D TP divisibility in the deployment script before launching instances","Prefer power-of-two TP sizes cluster-wide","Include a config lint step in CI for disagg topologies"],"tags":["tensor-parallel","kv-transfer","heterogeneous-tp","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}