Lightning-AI/pytorch-lightning · critical · RuntimeError

Training with multiple optimizers is only supported with man

Error message

Training with multiple optimizers is only supported with manual optimization. Set `self.automatic_optimization = False`, then access your optimizers in `training_step` with `opt1, opt2, ... = self.optimizers()`.

What it means

Returning more than one optimizer from configure_optimizers while self.automatic_optimization is True is unsupported in Lightning 2.x; Lightning raises RuntimeError telling you to enable manual optimization and step each optimizer yourself.

Source

Thrown at src/lightning/pytorch/core/optimizer.py:359

            and not is_overridden("lr_scheduler_step", model)
            and model.automatic_optimization
        ):
            raise MisconfigurationException(
                f"The provided lr scheduler `{scheduler.__class__.__name__}` doesn't follow PyTorch's LRScheduler"
                " API. You should override the `LightningModule.lr_scheduler_step` hook with your own logic if"
                " you are using a custom LR scheduler."
            )


def _validate_multiple_optimizers_support(optimizers: list[Optimizer], model: "pl.LightningModule") -> None:
    if is_param_in_hook_signature(model.training_step, "optimizer_idx", explicit=True):
        raise RuntimeError(
            "Training with multiple optimizers is only supported with manual optimization. Remove the `optimizer_idx`"
            " argument from `training_step`, set `self.automatic_optimization = False` and access your optimizers"
            " in `training_step` with `opt1, opt2, ... = self.optimizers()`."
        )
    if model.automatic_optimization and len(optimizers) > 1:
        raise RuntimeError(
            "Training with multiple optimizers is only supported with manual optimization. Set"
            " `self.automatic_optimization = False`, then access your optimizers in `training_step` with"
            " `opt1, opt2, ... = self.optimizers()`."
        )


def _validate_optimizers_attached(optimizers: list[Optimizer], lr_scheduler_configs: list[LRSchedulerConfig]) -> None:
    for config in lr_scheduler_configs:
        if config.scheduler.optimizer not in optimizers:
            raise MisconfigurationException(
                "Some schedulers are attached with an optimizer that wasn't returned from `configure_optimizers`."
            )


def _validate_optim_conf(optim_conf: dict[str, Any]) -> None:
    valid_keys = {"optimizer", "lr_scheduler", "monitor"}
    extra_keys = optim_conf.keys() - valid_keys
    if extra_keys:

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Set self.automatic_optimization = False in the LightningModule __init__
2. In training_step, get optimizers with opt1, opt2 = self.optimizers() and run zero_grad/backward/step for each
3. If you only need one optimizer, return a single optimizer from configure_optimizers

Example fix

# before
# automatic_optimization default True, configure_optimizers returns [opt1, opt2]
# after
def __init__(self):
    super().__init__()
    self.automatic_optimization = False

def training_step(self, batch, batch_idx):
    opt_gen, opt_disc = self.optimizers()
    ...  # manual backward/step per optimizer

n = len(model.configure_optimizers()[0]) if isinstance(model.configure_optimizers(), (list, tuple)) else 1
if n > 1:
    assert model.automatic_optimization is False, "multi-optimizer requires manual optimization"

def multi_opt_ok(module) -> bool:
    return not (len(getattr(module, "_optimizers", [])) > 1 and module.automatic_optimization)

When it happens

Trigger: configure_optimizers returns [opt1, opt2] and the module leaves automatic_optimization at its default True.

Common situations: GANs, multi-head models, or meta-learning setups migrated from Lightning 1.x that relied on automatic multi-optimizer stepping with optimizer_idx.

Related errors

• When `optimizer.step(closure)` is called, the closure should
• Training with multiple optimizers is only supported with man
• An optimizer should be passed only once to the `setup` metho
• Automatic gradient accumulation and the `GradientAccumulatio
• A single `Optimizer` cannot have multiple parameter groups w

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/1615617c41f44cf2. Report an issue: GitHub.