vllm-project/vllm · error · ValueError

Offline data parallel mode is not supported/useful for dense

Error message

Offline data parallel mode is not supported/useful for dense models.

What it means

In the offline (SPMD) path, ParallelConfig falls back to VLLM_DP_SIZE / VLLM_DP_RANK env vars when DP was not given in engine args. Offline data-parallel replicas each load a full copy of the model, which only pays off for MoE models whose experts can be sharded; for dense models it is rejected as unsupported and useless.

Source

Thrown at vllm/config/parallel.py:900

                self.data_parallel_master_port = (
                    self._data_parallel_master_port_list.pop()
                )

            if not (0 <= self.data_parallel_rank < self.data_parallel_size):
                raise ValueError(
                    f"data_parallel_rank ({self.data_parallel_rank})"
                    f" must be in the range [0, {self.data_parallel_size})"
                )
        else:
            # Otherwise fall back to env vars (e.g. for offline SPMD case).
            self.data_parallel_size = envs.VLLM_DP_SIZE
            self.data_parallel_rank = envs.VLLM_DP_RANK
            self.data_parallel_rank_local = envs.VLLM_DP_RANK_LOCAL
            self.data_parallel_master_ip = envs.VLLM_DP_MASTER_IP
            self.data_parallel_master_port = envs.VLLM_DP_MASTER_PORT

            if self.data_parallel_size > 1 and self.is_moe_model is False:
                raise ValueError(
                    "Offline data parallel mode is not supported/useful"
                    " for dense models."
                )

        self.data_parallel_index = self.data_parallel_rank

        if self.distributed_executor_backend == "external_launcher":
            os.environ["VLLM_ENABLE_V1_MULTIPROCESSING"] = "0"
            logger.info("Disabling V1 multiprocessing for external launcher.")

        if self.distributed_executor_backend is None and self.world_size_across_dp > 1:
            # We use multiprocessing by default if world_size fits on the
            # current node and we aren't in a ray placement group.

            from vllm.v1.executor import ray_utils

            backend: DistributedExecutorBackend = "mp"
            ray_found = ray_utils.ray_is_available()

View on GitHub (pinned to c794754062)

Solutions

Unset the DP env vars for dense models: `unset VLLM_DP_SIZE VLLM_DP_RANK VLLM_DP_RANK_LOCAL VLLM_DP_MASTER_IP VLLM_DP_MASTER_PORT`.
Or pass --data-parallel-size in the engine args instead of relying on env-based SPMD mode.
Switch to a MoE (mixture-of-experts) model if offline DP is genuinely needed.

Example fix

# before
export VLLM_DP_SIZE=2
python offline_infer.py --model dense-llama  # raises
# after
unset VLLM_DP_SIZE VLLM_DP_RANK VLLM_DP_RANK_LOCAL VLLM_DP_MASTER_IP VLLM_DP_MASTER_PORT
python offline_infer.py --model dense-llama

Defensive patterns

Strategy: validation

Validate before calling

import os

def offline_dp_env_ok(is_moe_model: bool) -> bool:
    dp_size = int(os.environ.get("VLLM_DP_SIZE", "1"))
    return dp_size <= 1 or is_moe_model

Prevention

Scope DP env vars to the exact command (env VLLM_DP_SIZE=2 python ...) instead of exporting them in shell profiles.
Add an entrypoint guard that clears stale VLLM_DP_* variables when the model config has no experts.

When it happens

Trigger: Running offline LLM inference with envs VLLM_DP_SIZE > 1 set (e.g. exported by a launcher) on a dense model, while is_moe_model is False.

Common situations: Launcher scripts that always export VLLM_DP_SIZE; switching a workload from a MoE model to a dense model without clearing DP env vars; CI environments carrying stale VLLM_* variables.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/1d2d4f169824df39. Report an issue: GitHub.