{"record":{"id":"1d2d4f169824df39","repo":"vllm-project/vllm","slug":"offline-data-parallel-mode-is-not-supported-useful","errorCode":null,"errorMessage":"Offline data parallel mode is not supported/useful for dense models.","messagePattern":"Offline data parallel mode is not supported/useful for dense models\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/parallel.py","lineNumber":900,"sourceCode":"                self.data_parallel_master_port = (\n                    self._data_parallel_master_port_list.pop()\n                )\n\n            if not (0 <= self.data_parallel_rank < self.data_parallel_size):\n                raise ValueError(\n                    f\"data_parallel_rank ({self.data_parallel_rank})\"\n                    f\" must be in the range [0, {self.data_parallel_size})\"\n                )\n        else:\n            # Otherwise fall back to env vars (e.g. for offline SPMD case).\n            self.data_parallel_size = envs.VLLM_DP_SIZE\n            self.data_parallel_rank = envs.VLLM_DP_RANK\n            self.data_parallel_rank_local = envs.VLLM_DP_RANK_LOCAL\n            self.data_parallel_master_ip = envs.VLLM_DP_MASTER_IP\n            self.data_parallel_master_port = envs.VLLM_DP_MASTER_PORT\n\n            if self.data_parallel_size > 1 and self.is_moe_model is False:\n                raise ValueError(\n                    \"Offline data parallel mode is not supported/useful\"\n                    \" for dense models.\"\n                )\n\n        self.data_parallel_index = self.data_parallel_rank\n\n        if self.distributed_executor_backend == \"external_launcher\":\n            os.environ[\"VLLM_ENABLE_V1_MULTIPROCESSING\"] = \"0\"\n            logger.info(\"Disabling V1 multiprocessing for external launcher.\")\n\n        if self.distributed_executor_backend is None and self.world_size_across_dp > 1:\n            # We use multiprocessing by default if world_size fits on the\n            # current node and we aren't in a ray placement group.\n\n            from vllm.v1.executor import ray_utils\n\n            backend: DistributedExecutorBackend = \"mp\"\n            ray_found = ray_utils.ray_is_available()","sourceCodeStart":882,"sourceCodeEnd":918,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/parallel.py#L882-L918","documentation":"In the offline (SPMD) path, ParallelConfig falls back to VLLM_DP_SIZE / VLLM_DP_RANK env vars when DP was not given in engine args. Offline data-parallel replicas each load a full copy of the model, which only pays off for MoE models whose experts can be sharded; for dense models it is rejected as unsupported and useless.","triggerScenarios":"Running offline LLM inference with envs VLLM_DP_SIZE > 1 set (e.g. exported by a launcher) on a dense model, while is_moe_model is False.","commonSituations":"Launcher scripts that always export VLLM_DP_SIZE; switching a workload from a MoE model to a dense model without clearing DP env vars; CI environments carrying stale VLLM_* variables.","solutions":["Unset the DP env vars for dense models: `unset VLLM_DP_SIZE VLLM_DP_RANK VLLM_DP_RANK_LOCAL VLLM_DP_MASTER_IP VLLM_DP_MASTER_PORT`.","Or pass --data-parallel-size in the engine args instead of relying on env-based SPMD mode.","Switch to a MoE (mixture-of-experts) model if offline DP is genuinely needed."],"exampleFix":"# before\nexport VLLM_DP_SIZE=2\npython offline_infer.py --model dense-llama  # raises\n# after\nunset VLLM_DP_SIZE VLLM_DP_RANK VLLM_DP_RANK_LOCAL VLLM_DP_MASTER_IP VLLM_DP_MASTER_PORT\npython offline_infer.py --model dense-llama","handlingStrategy":"validation","validationCode":"import os\n\ndef offline_dp_env_ok(is_moe_model: bool) -> bool:\n    dp_size = int(os.environ.get(\"VLLM_DP_SIZE\", \"1\"))\n    return dp_size <= 1 or is_moe_model","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Scope DP env vars to the exact command (env VLLM_DP_SIZE=2 python ...) instead of exporting them in shell profiles.","Add an entrypoint guard that clears stale VLLM_DP_* variables when the model config has no experts."],"tags":["offline-inference","data-parallel","environment-variables","dense-model"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}