{"record":{"id":"1f33f70b7cd5216d","repo":"vllm-project/vllm","slug":"number-of-experts-in-the-model-must-be-greater-tha","errorCode":null,"errorMessage":"Number of experts in the model must be greater than 0 when expert parallelism is enabled.","messagePattern":"Number of experts in the model must be greater than 0 when expert parallelism is enabled\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/model.py","lineNumber":1316,"sourceCode":"                    \"and will be removed in future versions of vLLM. To bypass, \"\n                    \"set `--allow-deprecated-quantization`.\",\n                    self.quantization,\n                )\n\n    def _verify_cuda_graph(self) -> None:\n        # CUDAGraph capture not supported for encoder-decoder models on ROCm\n        unsupported_rocm = self.is_encoder_decoder\n        if unsupported_rocm and not self.enforce_eager and current_platform.is_rocm():\n            logger.warning(\n                \"CUDA graph is not supported for %s on ROCm yet, fallback \"\n                \"to eager mode.\",\n                self.model_arch_config.model_type,\n            )\n            self.enforce_eager = True\n\n    def _verify_with_expert_parallelism(self) -> None:\n        if not self.is_moe:\n            raise ValueError(\n                \"Number of experts in the model must be greater than 0 \"\n                \"when expert parallelism is enabled.\"\n            )\n\n    def _try_verify_and_update_model_config(self):\n        # Avoid running try_verify_and_update_config multiple times\n        if getattr(self, \"config_updated\", False):\n            return\n\n        architecture = self.architecture\n        if architecture is None:\n            return\n\n        from vllm.model_executor.models.config import (\n            MODELS_CONFIG_MAP,\n        )\n\n        cls = MODELS_CONFIG_MAP.get(architecture, None)","sourceCodeStart":1298,"sourceCodeEnd":1334,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/model.py#L1298-L1334","documentation":"Raised by _verify_with_expert_parallelism when expert parallelism is enabled (--enable-expert-parallel) but the loaded model is not a Mixture-of-Experts model (is_moe is false, i.e. num routed experts is 0). Expert parallelism shards experts across ranks, so it is meaningless for dense models.","triggerScenarios":"ModelConfig.verify_with_parallel_config() runs with parallel_config.enable_expert_parallel=True and the model architecture/config reports no experts (self.is_moe is False).","commonSituations":"Copy-pasting a launch command from a MoE model (e.g. DeepSeek-V3, Mixtral) to a dense model (e.g. Llama) and leaving --enable-expert-parallel in the flags; enabling EP globally in a fleet config applied to mixed dense/MoE checkpoints.","solutions":["Remove --enable-expert-parallel (set enable_expert_parallel=False) if the model is dense.","If you intended expert parallelism, verify you loaded a MoE architecture and that its num_experts config field is parsed correctly (check the HF config / model type).","Confirm the correct checkpoint path — pointing at a dense sibling of a MoE model produces this."],"exampleFix":"# before\nvllm serve meta-llama/Llama-3.1-8B-Instruct --enable-expert-parallel\n# after\nvllm serve meta-llama/Llama-3.1-8B-Instruct","handlingStrategy":"validation","validationCode":"def validate_expert_parallel(model_config, parallel_config):\n    if parallel_config.enable_expert_parallel and not model_config.is_moe:\n        raise ConfigError('enable_expert_parallel set on dense model; disable it')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only set --enable-expert-parallel for architectures with routed experts (num_experts > 0 in HF config).","Keep per-model launch configs instead of one global flag set applied to all checkpoints.","Read the model's config.json num_experts before copying MoE serving flags."],"tags":["parallelism","moe","config","startup"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}