vllm-project/vllm · error · ValueError

Expert parallelism load balancing is only supported on CUDA

Error message

Expert parallelism load balancing is only supported on CUDA devices or ROCm devices now.

What it means

Expert parallelism load balancing (EPLB) is implemented only for CUDA and ROCm platforms. ParallelConfig calls current_platform.is_cuda_alike() and raises when EPLB is requested on any other backend (CPU, HPU, TPU, NPU, etc.).

Source

Thrown at vllm/config/parallel.py:495

                f"data_parallel_size_local ({self.data_parallel_size_local}) "
                f"must be <= data_parallel_size ({self.data_parallel_size})"
            )

        if self.data_parallel_size <= 1 and self.data_parallel_external_lb:
            raise ValueError(
                "data_parallel_external_lb can only be set when data_parallel_size > 1"
            )

        if not self.numa_bind and (
            self.numa_bind_nodes is not None or self.numa_bind_cpus is not None
        ):
            raise ValueError(
                "numa_bind_nodes and numa_bind_cpus require numa_bind=True."
            )

        if self.enable_eplb:
            if not current_platform.is_cuda_alike():
                raise ValueError(
                    "Expert parallelism load balancing is only supported on "
                    "CUDA devices or ROCm devices now."
                )
            if not self.enable_expert_parallel:
                raise ValueError("enable_expert_parallel must be True to use EPLB.")
            # The EP group spans the TP x PCP x DP ranks. EPLB therefore needs
            # TP, PCP, or DP > 1.
            if (
                self.tensor_parallel_size
                * self.prefill_context_parallel_size
                * self.data_parallel_size
                <= 1
            ):
                raise ValueError(
                    "EPLB requires tensor, prefill-context, or data parallelism, "
                    f"but got TP={self.tensor_parallel_size}, "
                    f"PCP={self.prefill_context_parallel_size}, "
                    f"DP={self.data_parallel_size}."

View on GitHub (pinned to c794754062)

Solutions

1. Run the EPLB deployment on CUDA or ROCm GPUs.
2. Or remove --enable-eplb (and any eplb_config with num_redundant_experts != 0) when serving on other platforms.
3. Gate the flag by detected platform in your launch script so CPU/NPU runs never pass it.

Example fix

# before
vllm serve Qwen3-MoE --enable-eplb   # on CPU
# after
vllm serve Qwen3-MoE                    # on CPU; keep --enable-eplb only on CUDA/ROCm

from vllm.platforms import current_platform

def eplb_supported() -> bool:
    return current_platform.is_cuda_alike()

When it happens

Trigger: Running --enable-eplb on a non-CUDA/ROCm platform, e.g. CPU-only container, Intel Gaudi, or Ascend NPU.

Common situations: Config template written for GPU clusters reused on alternative accelerators; CI sanity job on CPU-only runners inheriting the full production flag set.

Related errors

• enable_expert_parallel must be True to use EPLB.
• EPLB requires tensor, prefill-context, or data parallelism,
• num_redundant_experts is set to {self.eplb_config.num_redund
• Elastic EP is only supported with enable_eplb=True.
• multimodal preprocessing error: {0}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/8816120269ddb3de. Report an issue: GitHub.