vllm-project/vllm · critical · ValueError

nnodes > 1 can only be set when distributed executor backend

Error message

nnodes > 1 can only be set when distributed executor backend is mp, uni or external_launcher.

What it means

ParallelConfig rejects nnodes > 1 when the chosen distributed executor backend is not one of 'mp', 'uni', or 'external_launcher'. Multi-node execution in vLLM is only supported by those backends; Ray-based multi-node is configured through Ray itself (data_parallel_backend='ray'), not via nnodes, and other custom backends declare no multi-node support.

Source

Thrown at vllm/config/parallel.py:968

                        if get_current_placement_group():
                            backend = "ray"
            self.distributed_executor_backend = backend
            logger.debug("Defaulting to use %s for distributed inference", backend)

        if self.distributed_executor_backend is None and self.world_size == 1:
            self.distributed_executor_backend = "uni"

        if self.max_parallel_loading_workers is not None:
            logger.warning(
                "max_parallel_loading_workers is currently "
                "not supported and will be ignored."
            )
        allowed_backends = ("mp", "uni", "external_launcher")
        if (
            self.distributed_executor_backend not in allowed_backends
            and self.nnodes > 1
        ):
            raise ValueError(
                "nnodes > 1 can only be set when distributed executor "
                "backend is mp, uni or external_launcher."
            )

        if self.enable_eplb and self.eplb_config.communicator is None:
            # Prefer NIXL when available: zero-copy RDMA reads, compatible
            # with both async EPLB and elastic EP (deferred remote setup).
            # Fallbacks: pynccl for elastic EP (stateless groups need it),
            # torch_gloo for static EP.  torch_nccl is avoided because NCCL
            # is incompatible with async EPLB (multi-stream conflicts) and
            # batched isend/irecv hangs under high load.
            # See https://github.com/pytorch/pytorch/issues/174288
            from vllm.distributed.nixl_utils import is_nixl_available

            if is_nixl_available():
                self.eplb_config.communicator = "nixl"
            elif self.enable_elastic_ep:
                self.eplb_config.communicator = "pynccl"

View on GitHub (pinned to c794754062)

Solutions

If using Ray for multi-node, drop --nnodes and let Ray manage placement: set --data-parallel-backend ray (or distributed_executor_backend ray) without nnodes.
If using multiprocessing multi-node, keep --distributed-executor-backend mp (default) with --nnodes/--node-rank.
If launching under an external launcher (e.g. torchrun/srun), use --distributed-executor-backend external_launcher with --nnodes.
For TPU SPMD, use backend 'uni' with nnodes.

Example fix

# before
--distributed-executor-backend ray --nnodes 2 --node-rank 0

# after (Ray manages nodes itself)
--distributed-executor-backend ray --tensor-parallel-size 2 --data-parallel-size 4

Defensive patterns

Strategy: validation

Validate before calling

ALLOWED_MULTI_NODE = {"mp", "uni", "external_launcher"}

def validate_multi_node(backend: str | None, nnodes: int) -> None:
    if nnodes > 1 and backend not in ALLOWED_MULTI_NODE:
        raise SystemExit(
            f"nnodes={nnodes} requires backend in {ALLOWED_MULTI_NODE}; "
            "for Ray multi-node drop --nnodes and set data_parallel_backend=ray."
        )

Prevention

Never combine --distributed-executor-backend ray with --nnodes; Ray schedules nodes itself.
Centralize executor-backend choice in one launch helper so backend/nnodes pairs are always consistent.

When it happens

Trigger: Passing --nnodes 2 (or --data-parallel-size with nnodes>1) together with a distributed_executor_backend outside ('mp','uni','external_launcher'), e.g. backend='ray' with nnodes=2, or a custom Executor subclass with nnodes>1.

Common situations: User upgrades from an older vLLM where nnodes+ray was accepted; user mixes --distributed-executor-backend ray with --nnodes expecting Ray to honor it; scripts migrated from torchrun-style launchers keep --nnodes while switching backends.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/6848d57736260c1d. Report an issue: GitHub.