Lightning-AI/pytorch-lightning · error · MisconfigurationException

`model.configure_optimizers()` returned {len(optimizers)}, b

Error message

`model.configure_optimizers()` returned {len(optimizers)}, but learning rate finder only works with single optimizer

What it means

The LR finder needs to swap the optimizer's scheduler for its sweep scheduler, but the Trainer's strategy holds a number of optimizers other than one (model.configure_optimizers() returned 0 or 2+). LR range search only works with a single optimizer.

Source

Thrown at src/lightning/pytorch/tuner/lr_finder.py:100

        self.mode = mode
        self.lr_min = lr_min
        self.lr_max = lr_max
        self.num_training = num_training

        self.results: dict[str, Any] = {}
        self._total_batch_idx = 0  # for debug purpose

    def _exchange_scheduler(self, trainer: "pl.Trainer") -> None:
        # TODO: update docs here
        """Decorate `trainer.strategy.setup_optimizers` method such that it sets the user's originally specified
        optimizer together with a new scheduler that takes care of the learning rate search."""
        from lightning.pytorch.core.optimizer import _validate_optimizers_attached

        optimizers = trainer.strategy.optimizers

        if len(optimizers) != 1:
            raise MisconfigurationException(
                f"`model.configure_optimizers()` returned {len(optimizers)}, but"
                " learning rate finder only works with single optimizer"
            )

        optimizer = optimizers[0]

        new_lrs = [self.lr_min] * len(optimizer.param_groups)
        for param_group, new_lr in zip(optimizer.param_groups, new_lrs):
            param_group["lr"] = new_lr
            param_group["initial_lr"] = new_lr

        args = (optimizer, self.lr_max, self.num_training)
        scheduler = _LinearLR(*args) if self.mode == "linear" else _ExponentialLR(*args)

        trainer.strategy.optimizers = [optimizer]
        trainer.strategy.lr_scheduler_configs = [LRSchedulerConfig(scheduler, interval="step")]
        _validate_optimizers_attached(trainer.optimizers, trainer.lr_scheduler_configs)

View on GitHub (pinned to 9fed5c27d2)

Solutions

Restructure so configure_optimizers returns exactly one optimizer for the finder run
Run lr_find on a simplified variant of the model with a single optimizer
Manually sweep the LR with a loop of short fit runs if multiple optimizers are required

Example fix

# before (two optimizers -> lr_find fails)
def configure_optimizers(self):
    return [self.opt_g, self.opt_d], [sched_g, sched_d]
# after (single optimizer for lr_find)
def configure_optimizers(self):
    return torch.optim.Adam(self.parameters(), lr=self.lr)

Defensive patterns

Strategy: validation

Validate before calling

n = 1  # run a one-batch fit so optimizers are created, then:
opts = trainer.strategy.optimizers
if len(opts) != 1:
    raise ValueError(f"lr_find needs exactly 1 optimizer, found {len(opts)}")

Prevention

Run lr_find on single-optimizer models or a single-optimizer variant
Refactor multi-optimizer configure_optimizers into a switchable mode for tuning

When it happens

Trigger: tuner.lr_find(model) where configure_optimizers returns multiple optimizers (e.g. GAN with generator+discriminator optimizers) or returns None/empty.

Common situations: Running the LR finder on models with multiple optimizers (GANs, adversarial training) or where configure_optimizers failed to return anything before lr_find was called.

Related errors

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/4bf996e49d2f4319. Report an issue: GitHub.