{"record":{"id":"9206cb6401c6019f","repo":"vllm-project/vllm","slug":"data-parallel-rank-self-data-parallel-rank-mus","errorCode":null,"errorMessage":"data_parallel_rank ({self.data_parallel_rank}) must be in the range [0, {self.data_parallel_size})","messagePattern":"data_parallel_rank \\((.+?)\\) must be in the range \\[0, (.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/parallel.py","lineNumber":887,"sourceCode":"            if self.distributed_executor_backend == \"external_launcher\":\n                # For external launcher,\n                # we need to set the data parallel rank automatically\n                self.data_parallel_rank = int(os.environ[\"RANK\"]) // (\n                    self.world_size // self.data_parallel_size\n                )\n                logger.info(\n                    \"Set data_parallel_rank to %d automatically.\",\n                    self.data_parallel_rank,\n                )\n            if not self.enable_elastic_ep:\n                if not self._data_parallel_master_port_list:\n                    self._data_parallel_master_port_list = get_open_ports_list(5)\n                self.data_parallel_master_port = (\n                    self._data_parallel_master_port_list.pop()\n                )\n\n            if not (0 <= self.data_parallel_rank < self.data_parallel_size):\n                raise ValueError(\n                    f\"data_parallel_rank ({self.data_parallel_rank})\"\n                    f\" must be in the range [0, {self.data_parallel_size})\"\n                )\n        else:\n            # Otherwise fall back to env vars (e.g. for offline SPMD case).\n            self.data_parallel_size = envs.VLLM_DP_SIZE\n            self.data_parallel_rank = envs.VLLM_DP_RANK\n            self.data_parallel_rank_local = envs.VLLM_DP_RANK_LOCAL\n            self.data_parallel_master_ip = envs.VLLM_DP_MASTER_IP\n            self.data_parallel_master_port = envs.VLLM_DP_MASTER_PORT\n\n            if self.data_parallel_size > 1 and self.is_moe_model is False:\n                raise ValueError(\n                    \"Offline data parallel mode is not supported/useful\"\n                    \" for dense models.\"\n                )\n\n        self.data_parallel_index = self.data_parallel_rank","sourceCodeStart":869,"sourceCodeEnd":905,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/parallel.py#L869-L905","documentation":"After resolving data_parallel_rank (from args or from RANK for external launchers), ParallelConfig asserts 0 <= data_parallel_rank < data_parallel_size. An out-of-range rank means the process would map to a nonexistent DP slot.","triggerScenarios":"Explicitly passing --data-parallel-rank R with R >= data_parallel_size or negative; or with external_launcher, when env RANK // (world_size // dp_size) lands outside [0, dp_size) due to mismatched RANK/WORLD_SIZE/DP settings.","commonSituations":"Mismatched torchrun RANK/WORLD_SIZE versus --data-parallel-size in custom launch scripts; manually editing rank values when debugging a multi-node job; off-by-one errors computing per-node ranks.","solutions":["Set --data-parallel-rank to a value in [0, data_parallel_size), e.g. 0 or 1 for dp_size=2.","For external_launcher, recheck that RANK/WORLD_SIZE env vars and --data-parallel-size are consistent (world_size == dp_size * tp * pcp * pp).","Let vLLM derive the rank automatically (omit --data-parallel-rank) in supported launch modes."],"exampleFix":"# before\nvllm serve model --data-parallel-size 2 --data-parallel-rank 2\n# after\nvllm serve model --data-parallel-size 2 --data-parallel-rank 1","handlingStrategy":"validation","validationCode":"def dp_rank_valid(dp_rank: int, dp_size: int) -> bool:\n    return 0 <= dp_rank < dp_size\n\nassert dp_rank_valid(1, 2)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["For external_launcher, assert RANK < WORLD_SIZE == dp_size * world_size before starting vLLM.","Prefer letting vLLM derive data_parallel_rank instead of passing it manually."],"tags":["data-parallel","multi-node","launcher","configuration"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}