{"record":{"id":"946da0b106720962","repo":"vllm-project/vllm","slug":"data-parallel-size-local-self-data-parallel-size","errorCode":null,"errorMessage":"data_parallel_size_local ({self.data_parallel_size_local}) must be <= data_parallel_size ({self.data_parallel_size})","messagePattern":"data_parallel_size_local \\((.+?)\\) must be <= data_parallel_size \\((.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/parallel.py","lineNumber":476,"sourceCode":"            )\n\n        if self.enable_fault_tolerance and self._api_process_count > 1:\n            raise ValueError(\n                \"Fault tolerance requires a single API server process \"\n                f\"(--api-server-count=1), but got {self._api_process_count}. \"\n                \"The FT system assumes one AsyncMPClient manages all engines.\"\n            )\n\n        if self.all2all_backend in [\"pplx\", \"naive\"]:\n            logger.warning(\n                \"The '%s' all2all backend has been removed. \"\n                \"Falling back to 'allgather_reducescatter'.\",\n                self.all2all_backend,\n            )\n            self.all2all_backend = \"allgather_reducescatter\"\n\n        if self.data_parallel_size_local > self.data_parallel_size:\n            raise ValueError(\n                f\"data_parallel_size_local ({self.data_parallel_size_local}) \"\n                f\"must be <= data_parallel_size ({self.data_parallel_size})\"\n            )\n\n        if self.data_parallel_size <= 1 and self.data_parallel_external_lb:\n            raise ValueError(\n                \"data_parallel_external_lb can only be set when data_parallel_size > 1\"\n            )\n\n        if not self.numa_bind and (\n            self.numa_bind_nodes is not None or self.numa_bind_cpus is not None\n        ):\n            raise ValueError(\n                \"numa_bind_nodes and numa_bind_cpus require numa_bind=True.\"\n            )\n\n        if self.enable_eplb:\n            if not current_platform.is_cuda_alike():","sourceCodeStart":458,"sourceCodeEnd":494,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/parallel.py#L458-L494","documentation":"data_parallel_size_local bounds how many of the data-parallel ranks run on this node. ParallelConfig rejects it when it exceeds the total data_parallel_size, since a per-node count cannot be larger than the global count.","triggerScenarios":"Passing --data-parallel-size-local greater than --data-parallel-size, e.g. local=4 with size=2.","commonSituations":"Editing a multi-node launch script and shrinking --data-parallel-size without updating the local value; assuming data_parallel_size_local is an independent knob rather than a sub-count of the global DP size.","solutions":["Raise --data-parallel-size to at least the local value, e.g. --data-parallel-size 4 --data-parallel-size-local 4.","Or lower --data-parallel-size-local so it is <= data_parallel_size.","Leave data_parallel_size_local unset (0) on single-node setups so vLLM treats all DP ranks as local."],"exampleFix":"# before\n--data-parallel-size 2 --data-parallel-size-local 4\n# after\n--data-parallel-size 4 --data-parallel-size-local 4","handlingStrategy":"validation","validationCode":"def dp_sizes_valid(dp_size: int, dp_size_local: int) -> bool:\n    return dp_size_local <= dp_size\n\nassert dp_sizes_valid(4, 4)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["On single nodes, leave --data-parallel-size-local unset.","Derive dp_size from sum of per-node local values in multi-node scripts so they cannot diverge."],"tags":["data-parallel","configuration","multi-node","parallelism"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}