vllm-project/vllm · critical · ValueError

World size ({self.world_size}) is larger than the number of

Error message

World size ({self.world_size}) is larger than the number of available GPUs ({gpu_count}) in this node. If this is intentional and you are using:
- ray, set '--distributed-executor-backend ray'.
- multiprocessing, set '--nnodes' appropriately.

What it means

Raised by ParallelConfig during startup when CUDA is the platform and the computed world size (tensor_parallel_size * data_parallel_size_local, etc.) exceeds the number of GPUs visible on this node. The validation happens in the auto-backend-selection path, so it fires when distributed_executor_backend was left to auto-detect. vLLM refuses to start because multiprocessing-style workers cannot be placed on GPUs that do not exist.

Source

Thrown at vllm/config/parallel.py:928

        if self.distributed_executor_backend is None and self.world_size_across_dp > 1:
            # We use multiprocessing by default if world_size fits on the
            # current node and we aren't in a ray placement group.

            from vllm.v1.executor import ray_utils

            backend: DistributedExecutorBackend = "mp"
            ray_found = ray_utils.ray_is_available()
            if current_platform.is_tpu() and envs.VLLM_XLA_USE_SPMD:
                backend = "uni"
            elif current_platform.is_cuda() and self.nnodes > 1:
                backend = "mp"
            elif (
                current_platform.is_cuda()
                and current_platform.device_count() < self.world_size
            ):
                gpu_count = current_platform.device_count()
                raise ValueError(
                    f"World size ({self.world_size}) is larger than the number of "
                    f"available GPUs ({gpu_count}) in this node. If this is "
                    "intentional and you are using:\n"
                    "- ray, set '--distributed-executor-backend ray'.\n"
                    "- multiprocessing, set '--nnodes' appropriately."
                )
            elif self.data_parallel_backend == "ray":
                logger.info(
                    "Using ray distributed inference because "
                    "data_parallel_backend is ray"
                )
                backend = "ray"
            elif ray_found:
                if self.placement_group:
                    backend = "ray"
                else:
                    from ray import is_initialized as ray_is_initialized

View on GitHub (pinned to c794754062)

Solutions

Reduce parallel size to fit: set --tensor-parallel-size (and pipeline/data-parallel sizes) so world size <= visible GPU count.
If using a multi-node Ray cluster, pass --distributed-executor-backend ray.
If truly running multi-node with multiprocessing, set --nnodes (plus --node-rank, --pipeline-parallel-size per node) appropriately.
Check CUDA_VISIBLE_DEVICES / nvidia-smi to confirm how many GPUs are actually visible to the process.

Example fix

# before
vllm serve meta-llama/Llama-3-70B --tensor-parallel-size 8   # box has 4 GPUs

# after
vllm serve meta-llama/Llama-3-70B --tensor-parallel-size 4
# or, on a Ray cluster:
vllm serve meta-llama/Llama-3-70B --tensor-parallel-size 8 --distributed-executor-backend ray

Defensive patterns

Strategy: validation

Validate before calling

import torch
from vllm.config import ParallelConfig

def check_world_size_fits(tp: int, pp: int = 1, dp_local: int = 1) -> None:
    gpu_count = torch.cuda.device_count() if torch.cuda.is_available() else 0
    world_size = tp * pp * dp_local
    if world_size > gpu_count:
        raise SystemExit(
            f"world_size={world_size} exceeds visible GPUs={gpu_count}; "
            "lower parallel sizes, fix CUDA_VISIBLE_DEVICES, or use "
            "--distributed-executor-backend ray for multi-node."
        )

Try / catch

try:
    cfg = ParallelConfig(tensor_parallel_size=tp, ...)
except ValueError as e:
    if "larger than the number of available GPUs" in str(e):
        raise SystemExit(f"Config error: {e}; check CUDA_VISIBLE_DEVICES / parallel sizes")
    raise

Prevention

Print torch.cuda.device_count() at script start and assert parallel sizes divide into it.
Keep CUDA_VISIBLE_DEVICES explicit in launch scripts rather than inherited.
For multi-node, always pair --distributed-executor-backend ray with the parallel sizes instead of relying on auto-detection.

When it happens

Trigger: Setting --tensor-network-size / --tensor-parallel-size (or pipeline/data parallel sizes) larger than CUDA_VISIBLE_DEVICES count on a single node, while leaving distributed_executor_backend unset, on a CUDA (non-TPU, single-node) setup. E.g. tensor_parallel_size=8 on a 4-GPU box, or CUDA_VISIBLE_DEVICES=0 hiding all but one GPU.

Common situations: Developer copies a multi-GPU launch command onto a smaller dev box; CUDA_VISIBLE_DEVICES restricts visible devices; intends to use a Ray cluster spanning nodes but forgot to pass --distributed-executor-backend ray; GPU drivers/NCCL issues make device_count() report fewer GPUs.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/eaccac20346be48d. Report an issue: GitHub.