{"record":{"id":"4bf996e49d2f4319","repo":"Lightning-AI/pytorch-lightning","slug":"model-configure-optimizers-returned-len-optim","errorCode":null,"errorMessage":"`model.configure_optimizers()` returned {len(optimizers)}, but learning rate finder only works with single optimizer","messagePattern":"`model\\.configure_optimizers\\(\\)` returned (.+?), but learning rate finder only works with single optimizer","errorType":"validation","errorClass":"MisconfigurationException","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/tuner/lr_finder.py","lineNumber":100,"sourceCode":"\n        self.mode = mode\n        self.lr_min = lr_min\n        self.lr_max = lr_max\n        self.num_training = num_training\n\n        self.results: dict[str, Any] = {}\n        self._total_batch_idx = 0  # for debug purpose\n\n    def _exchange_scheduler(self, trainer: \"pl.Trainer\") -> None:\n        # TODO: update docs here\n        \"\"\"Decorate `trainer.strategy.setup_optimizers` method such that it sets the user's originally specified\n        optimizer together with a new scheduler that takes care of the learning rate search.\"\"\"\n        from lightning.pytorch.core.optimizer import _validate_optimizers_attached\n\n        optimizers = trainer.strategy.optimizers\n\n        if len(optimizers) != 1:\n            raise MisconfigurationException(\n                f\"`model.configure_optimizers()` returned {len(optimizers)}, but\"\n                \" learning rate finder only works with single optimizer\"\n            )\n\n        optimizer = optimizers[0]\n\n        new_lrs = [self.lr_min] * len(optimizer.param_groups)\n        for param_group, new_lr in zip(optimizer.param_groups, new_lrs):\n            param_group[\"lr\"] = new_lr\n            param_group[\"initial_lr\"] = new_lr\n\n        args = (optimizer, self.lr_max, self.num_training)\n        scheduler = _LinearLR(*args) if self.mode == \"linear\" else _ExponentialLR(*args)\n\n        trainer.strategy.optimizers = [optimizer]\n        trainer.strategy.lr_scheduler_configs = [LRSchedulerConfig(scheduler, interval=\"step\")]\n        _validate_optimizers_attached(trainer.optimizers, trainer.lr_scheduler_configs)\n","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/tuner/lr_finder.py#L82-L118","documentation":"The LR finder needs to swap the optimizer's scheduler for its sweep scheduler, but the Trainer's strategy holds a number of optimizers other than one (model.configure_optimizers() returned 0 or 2+). LR range search only works with a single optimizer.","triggerScenarios":"tuner.lr_find(model) where configure_optimizers returns multiple optimizers (e.g. GAN with generator+discriminator optimizers) or returns None/empty.","commonSituations":"Running the LR finder on models with multiple optimizers (GANs, adversarial training) or where configure_optimizers failed to return anything before lr_find was called.","solutions":["Restructure so configure_optimizers returns exactly one optimizer for the finder run","Run lr_find on a simplified variant of the model with a single optimizer","Manually sweep the LR with a loop of short fit runs if multiple optimizers are required"],"exampleFix":"# before (two optimizers -> lr_find fails)\ndef configure_optimizers(self):\n    return [self.opt_g, self.opt_d], [sched_g, sched_d]\n# after (single optimizer for lr_find)\ndef configure_optimizers(self):\n    return torch.optim.Adam(self.parameters(), lr=self.lr)","handlingStrategy":"validation","validationCode":"n = 1  # run a one-batch fit so optimizers are created, then:\nopts = trainer.strategy.optimizers\nif len(opts) != 1:\n    raise ValueError(f\"lr_find needs exactly 1 optimizer, found {len(opts)}\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Run lr_find on single-optimizer models or a single-optimizer variant","Refactor multi-optimizer configure_optimizers into a switchable mode for tuning"],"tags":["lr-finder","tuner","multiple-optimizers","configure-optimizers","pytorch-lightning"],"backgroundTag":"multiple-optimizers-unsupported","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}