Lightning-AI/pytorch-lightning · critical · RuntimeError

Training with multiple optimizers is only supported with man

Error message

Training with multiple optimizers is only supported with manual optimization. Remove the `optimizer_idx` argument from `training_step`, set `self.automatic_optimization = False` and access your optimizers in `training_step` with `opt1, opt2, ... = self.optimizers()`.

What it means

The `optimizer_idx` argument in training_step belonged to the removed automatic multi-optimizer API. If Lightning detects it in the training_step signature it raises RuntimeError instructing you to move to manual optimization and fetch optimizers via self.optimizers().

Source

Thrown at src/lightning/pytorch/core/optimizer.py:353

                f"The provided lr scheduler `{scheduler.__class__.__name__}` is invalid."
                " It should have `state_dict` and `load_state_dict` methods defined."
            )

        if (
            not isinstance(scheduler, LRSchedulerTypeTuple)
            and not is_overridden("lr_scheduler_step", model)
            and model.automatic_optimization
        ):
            raise MisconfigurationException(
                f"The provided lr scheduler `{scheduler.__class__.__name__}` doesn't follow PyTorch's LRScheduler"
                " API. You should override the `LightningModule.lr_scheduler_step` hook with your own logic if"
                " you are using a custom LR scheduler."
            )


def _validate_multiple_optimizers_support(optimizers: list[Optimizer], model: "pl.LightningModule") -> None:
    if is_param_in_hook_signature(model.training_step, "optimizer_idx", explicit=True):
        raise RuntimeError(
            "Training with multiple optimizers is only supported with manual optimization. Remove the `optimizer_idx`"
            " argument from `training_step`, set `self.automatic_optimization = False` and access your optimizers"
            " in `training_step` with `opt1, opt2, ... = self.optimizers()`."
        )
    if model.automatic_optimization and len(optimizers) > 1:
        raise RuntimeError(
            "Training with multiple optimizers is only supported with manual optimization. Set"
            " `self.automatic_optimization = False`, then access your optimizers in `training_step` with"
            " `opt1, opt2, ... = self.optimizers()`."
        )


def _validate_optimizers_attached(optimizers: list[Optimizer], lr_scheduler_configs: list[LRSchedulerConfig]) -> None:
    for config in lr_scheduler_configs:
        if config.scheduler.optimizer not in optimizers:
            raise MisconfigurationException(
                "Some schedulers are attached with an optimizer that wasn't returned from `configure_optimizers`."
            )

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Remove the optimizer_idx parameter from training_step
2. Set self.automatic_optimization = False in __init__
3. Access optimizers in training_step via opt1, opt2 = self.optimizers() and call opt.zero_grad(); loss.backward(); opt.step() manually

Example fix

# before
def training_step(self, batch, batch_idx, optimizer_idx):
    ...
# after
def __init__(self):
    self.automatic_optimization = False

def training_step(self, batch, batch_idx):
    opt1, opt2 = self.optimizers()
    ...

import inspect
sig = inspect.signature(model.training_step)
assert "optimizer_idx" not in sig.parameters, "optimizer_idx removed in Lightning 2.x"

def has_optimizer_idx(module) -> bool:
    import inspect
    return "optimizer_idx" in inspect.signature(module.training_step).parameters

When it happens

Trigger: Defining def training_step(self, batch, batch_idx, optimizer_idx) in a LightningModule, regardless of optimizer count.

Common situations: Upgrading code from Lightning 1.x to 2.x where optimizer_idx in automatic optimization was removed.

Related errors

• When `optimizer.step(closure)` is called, the closure should
• Training with multiple optimizers is only supported with man
• An optimizer should be passed only once to the `setup` metho
• Automatic gradient accumulation and the `GradientAccumulatio
• A single `Optimizer` cannot have multiple parameter groups w

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/33a2050b2631c679. Report an issue: GitHub.