{"record":{"id":"eaccac20346be48d","repo":"vllm-project/vllm","slug":"world-size-self-world-size-is-larger-than-the","errorCode":null,"errorMessage":"World size ({self.world_size}) is larger than the number of available GPUs ({gpu_count}) in this node. If this is intentional and you are using:\n- ray, set '--distributed-executor-backend ray'.\n- multiprocessing, set '--nnodes' appropriately.","messagePattern":"World size \\((.+?)\\) is larger than the number of available GPUs \\((.+?)\\) in this node\\. If this is intentional and you are using:\n- ray, set '--distributed-executor-backend ray'\\.\n- multiprocessing, set '--nnodes' appropriately\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"vllm/config/parallel.py","lineNumber":928,"sourceCode":"\n        if self.distributed_executor_backend is None and self.world_size_across_dp > 1:\n            # We use multiprocessing by default if world_size fits on the\n            # current node and we aren't in a ray placement group.\n\n            from vllm.v1.executor import ray_utils\n\n            backend: DistributedExecutorBackend = \"mp\"\n            ray_found = ray_utils.ray_is_available()\n            if current_platform.is_tpu() and envs.VLLM_XLA_USE_SPMD:\n                backend = \"uni\"\n            elif current_platform.is_cuda() and self.nnodes > 1:\n                backend = \"mp\"\n            elif (\n                current_platform.is_cuda()\n                and current_platform.device_count() < self.world_size\n            ):\n                gpu_count = current_platform.device_count()\n                raise ValueError(\n                    f\"World size ({self.world_size}) is larger than the number of \"\n                    f\"available GPUs ({gpu_count}) in this node. If this is \"\n                    \"intentional and you are using:\\n\"\n                    \"- ray, set '--distributed-executor-backend ray'.\\n\"\n                    \"- multiprocessing, set '--nnodes' appropriately.\"\n                )\n            elif self.data_parallel_backend == \"ray\":\n                logger.info(\n                    \"Using ray distributed inference because \"\n                    \"data_parallel_backend is ray\"\n                )\n                backend = \"ray\"\n            elif ray_found:\n                if self.placement_group:\n                    backend = \"ray\"\n                else:\n                    from ray import is_initialized as ray_is_initialized\n","sourceCodeStart":910,"sourceCodeEnd":946,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/parallel.py#L910-L946","documentation":"Raised by ParallelConfig during startup when CUDA is the platform and the computed world size (tensor_parallel_size * data_parallel_size_local, etc.) exceeds the number of GPUs visible on this node. The validation happens in the auto-backend-selection path, so it fires when distributed_executor_backend was left to auto-detect. vLLM refuses to start because multiprocessing-style workers cannot be placed on GPUs that do not exist.","triggerScenarios":"Setting --tensor-network-size / --tensor-parallel-size (or pipeline/data parallel sizes) larger than CUDA_VISIBLE_DEVICES count on a single node, while leaving distributed_executor_backend unset, on a CUDA (non-TPU, single-node) setup. E.g. tensor_parallel_size=8 on a 4-GPU box, or CUDA_VISIBLE_DEVICES=0 hiding all but one GPU.","commonSituations":"Developer copies a multi-GPU launch command onto a smaller dev box; CUDA_VISIBLE_DEVICES restricts visible devices; intends to use a Ray cluster spanning nodes but forgot to pass --distributed-executor-backend ray; GPU drivers/NCCL issues make device_count() report fewer GPUs.","solutions":["Reduce parallel size to fit: set --tensor-parallel-size (and pipeline/data-parallel sizes) so world size <= visible GPU count.","If using a multi-node Ray cluster, pass --distributed-executor-backend ray.","If truly running multi-node with multiprocessing, set --nnodes (plus --node-rank, --pipeline-parallel-size per node) appropriately.","Check CUDA_VISIBLE_DEVICES / nvidia-smi to confirm how many GPUs are actually visible to the process."],"exampleFix":"# before\nvllm serve meta-llama/Llama-3-70B --tensor-parallel-size 8   # box has 4 GPUs\n\n# after\nvllm serve meta-llama/Llama-3-70B --tensor-parallel-size 4\n# or, on a Ray cluster:\nvllm serve meta-llama/Llama-3-70B --tensor-parallel-size 8 --distributed-executor-backend ray","handlingStrategy":"validation","validationCode":"import torch\nfrom vllm.config import ParallelConfig\n\ndef check_world_size_fits(tp: int, pp: int = 1, dp_local: int = 1) -> None:\n    gpu_count = torch.cuda.device_count() if torch.cuda.is_available() else 0\n    world_size = tp * pp * dp_local\n    if world_size > gpu_count:\n        raise SystemExit(\n            f\"world_size={world_size} exceeds visible GPUs={gpu_count}; \"\n            \"lower parallel sizes, fix CUDA_VISIBLE_DEVICES, or use \"\n            \"--distributed-executor-backend ray for multi-node.\"\n        )","typeGuard":null,"tryCatchPattern":"try:\n    cfg = ParallelConfig(tensor_parallel_size=tp, ...)\nexcept ValueError as e:\n    if \"larger than the number of available GPUs\" in str(e):\n        raise SystemExit(f\"Config error: {e}; check CUDA_VISIBLE_DEVICES / parallel sizes\")\n    raise","preventionTips":["Print torch.cuda.device_count() at script start and assert parallel sizes divide into it.","Keep CUDA_VISIBLE_DEVICES explicit in launch scripts rather than inherited.","For multi-node, always pair --distributed-executor-backend ray with the parallel sizes instead of relying on auto-detection."],"tags":["parallelism","gpu","configuration","startup"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}