vllm-project/vllm · error · ValueError

Number of experts in the model must be greater than 0 when e

Error message

Number of experts in the model must be greater than 0 when expert parallelism is enabled.

What it means

Raised by _verify_with_expert_parallelism when expert parallelism is enabled (--enable-expert-parallel) but the loaded model is not a Mixture-of-Experts model (is_moe is false, i.e. num routed experts is 0). Expert parallelism shards experts across ranks, so it is meaningless for dense models.

Source

Thrown at vllm/config/model.py:1316

                    "and will be removed in future versions of vLLM. To bypass, "
                    "set `--allow-deprecated-quantization`.",
                    self.quantization,
                )

    def _verify_cuda_graph(self) -> None:
        # CUDAGraph capture not supported for encoder-decoder models on ROCm
        unsupported_rocm = self.is_encoder_decoder
        if unsupported_rocm and not self.enforce_eager and current_platform.is_rocm():
            logger.warning(
                "CUDA graph is not supported for %s on ROCm yet, fallback "
                "to eager mode.",
                self.model_arch_config.model_type,
            )
            self.enforce_eager = True

    def _verify_with_expert_parallelism(self) -> None:
        if not self.is_moe:
            raise ValueError(
                "Number of experts in the model must be greater than 0 "
                "when expert parallelism is enabled."
            )

    def _try_verify_and_update_model_config(self):
        # Avoid running try_verify_and_update_config multiple times
        if getattr(self, "config_updated", False):
            return

        architecture = self.architecture
        if architecture is None:
            return

        from vllm.model_executor.models.config import (
            MODELS_CONFIG_MAP,
        )

        cls = MODELS_CONFIG_MAP.get(architecture, None)

View on GitHub (pinned to c794754062)

Solutions

Remove --enable-expert-parallel (set enable_expert_parallel=False) if the model is dense.
If you intended expert parallelism, verify you loaded a MoE architecture and that its num_experts config field is parsed correctly (check the HF config / model type).
Confirm the correct checkpoint path — pointing at a dense sibling of a MoE model produces this.

Example fix

# before
vllm serve meta-llama/Llama-3.1-8B-Instruct --enable-expert-parallel
# after
vllm serve meta-llama/Llama-3.1-8B-Instruct

Defensive patterns

Strategy: validation

Validate before calling

def validate_expert_parallel(model_config, parallel_config):
    if parallel_config.enable_expert_parallel and not model_config.is_moe:
        raise ConfigError('enable_expert_parallel set on dense model; disable it')

Prevention

Only set --enable-expert-parallel for architectures with routed experts (num_experts > 0 in HF config).
Keep per-model launch configs instead of one global flag set applied to all checkpoints.
Read the model's config.json num_experts before copying MoE serving flags.

When it happens

Trigger: ModelConfig.verify_with_parallel_config() runs with parallel_config.enable_expert_parallel=True and the model architecture/config reports no experts (self.is_moe is False).

Common situations: Copy-pasting a launch command from a MoE model (e.g. DeepSeek-V3, Mixtral) to a dense model (e.g. Llama) and leaving --enable-expert-parallel in the flags; enabling EP globally in a fleet config applied to mixed dense/MoE checkpoints.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/1f33f70b7cd5216d. Report an issue: GitHub.