huggingface/transformers · error · ValueError

Expert parallelism was requested…

Error message

Expert parallelism was requested (`enable_expert_parallel=True`), but `{self.__class__.__name__}` does not define an expert-parallel plan. Add a `base_model_ep_plan` to its config, or disable expert parallelism.

What it means

The tp_plan property on distributed model mixins switches to the expert-parallel plan when the config sets enable_expert_parallel=True. If the model class never declared an expert-parallel plan (base_model_ep_plan on the config / _ep_plan on the class), the property raises instead of silently falling back to regular TP, because MoE expert sharding requires explicit placement that cannot be inferred.

Solutions

Disable expert parallelism: set enable_expert_parallel=False (or omit it) in DistributedConfig and use plain TP.
Keep expert parallelism only on model integrations that declare base_model_ep_plan; upgrade transformers if a newer release added the plan for your model.
For your own MoE integration, declare base_model_ep_plan on the config and _ep_plan on the head class mapping expert modules to expert-parallel sharding.

Example fix

# before
cfg = DistributedConfig(tp_size=8, enable_expert_parallel=True)  # no EP plan -> raises

# after
cfg = DistributedConfig(tp_size=8, enable_expert_parallel=False)
# or use a model version that declares base_model_ep_plan

Defensive patterns

Strategy: validation

Validate before calling

ep_requested = getattr(model.config, "distributed_config", None)
if ep_requested is not None and ep_requested.enable_expert_parallel and not getattr(model, "_ep_plan", None):
    ep_requested.enable_expert_parallel = False  # or raise, depending on policy

Type guard

def supports_expert_parallel(model) -> bool:
    return bool(getattr(model, "_ep_plan", None))

Try / catch

try:
    plan = model.tp_plan
except ValueError as e:
    if "expert-parallel" in str(e):
        model.config.distributed_config.enable_expert_parallel = False
        plan = model.tp_plan  # plain TP plan now
    else:
        raise

Prevention

Only enable enable_expert_parallel for models that declare base_model_ep_plan.
Validate the flag against model support in your launcher before training.
Upgrade transformers when MoE EP support is added for your model.

When it happens

Trigger: Loading a MoE model with DistributedConfig(enable_expert_parallel=True) when the model integration lacks base_model_ep_plan; toggling enable_expert_parallel in a config JSON for a model that only defines standard tp plans; accessing model.tp_plan on such a model.

Common situations: Enabling expert parallelism on a MoE (e.g. Mixtral/Qwen-MoE family) whose version does not yet ship an EP plan; mixing generic TP configs with MoE-specific features; porting configs between models.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/d1feeba88b1ee3ab. Report an issue: GitHub.

Appendix: source

Thrown at src/transformers/distributed/mixin.py:93

            self._ep_plan.update(self.config.base_model_ep_plan or {})
            self._fsdp_plan.update(self.config.base_model_fsdp_plan or {})

        for name, module in self.named_children():
            if plan := getattr(module, "_ep_plan", None):
                self._ep_plan.update({f"{name}.{k}": v for k, v in plan.copy().items()})
            if plan := getattr(module, "_tp_plan", None):
                self._tp_plan.update({f"{name}.{k}": v for k, v in plan.copy().items()})
            if plan := getattr(module, "_pp_plan", None):
                self._pp_plan.update({f"{name}.{k}": v for k, v in plan.copy().items()})
            if plan := getattr(module, "_fsdp_plan", None):
                self._fsdp_plan.update({f"{name}.{k}": v for k, v in plan.copy().items()})

    @property
    def tp_plan(self) -> dict[str, str]:
        """The full tp plan for the model's modules."""
        if hasattr(self.config, "distributed_config") and self.config.distributed_config.enable_expert_parallel:
            if not self._ep_plan:
                raise ValueError(
                    f"Expert parallelism was requested (`enable_expert_parallel=True`), but "
                    f"`{self.__class__.__name__}` does not define an expert-parallel plan. Add a "
                    f"`base_model_ep_plan` to its config, or disable expert parallelism."
                )
            return self._ep_plan
        return self._tp_plan

    @property
    def fsdp_plan(self) -> dict[str, str]:
        return self._fsdp_plan

    @property
    def pp_plan(self) -> dict[str, tuple[str, str]]:
        return self._pp_plan

    @tp_plan.setter
    def tp_plan(self, plan: dict[str, str] | None):
        if plan is None:

View on GitHub (pinned to a597f97485)