{"record":{"id":"76e04e686198009a","repo":"Lightning-AI/pytorch-lightning","slug":"swa-currently-works-with-1-optimizer","errorCode":null,"errorMessage":"SWA currently works with 1 `optimizer`.","messagePattern":"SWA currently works with 1 `optimizer`\\.","errorType":"exception","errorClass":"MisconfigurationException","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/callbacks/stochastic_weight_avg.py","lineNumber":161,"sourceCode":"            return float(\"inf\")  # type: ignore[return-value]\n        return self._max_epochs - 1  # 0-based\n\n    @staticmethod\n    def pl_module_contains_batch_norm(pl_module: \"pl.LightningModule\") -> bool:\n        return any(isinstance(module, nn.modules.batchnorm._BatchNorm) for module in pl_module.modules())\n\n    @override\n    def setup(self, trainer: \"pl.Trainer\", pl_module: \"pl.LightningModule\", stage: str) -> None:\n        if isinstance(trainer.strategy, (FSDPStrategy, DeepSpeedStrategy)):\n            raise MisconfigurationException(\"SWA does not currently support sharded models.\")\n\n        # copy the model before moving it to accelerator device.\n        self._average_model = deepcopy(pl_module)\n\n    @override\n    def on_fit_start(self, trainer: \"pl.Trainer\", pl_module: \"pl.LightningModule\") -> None:\n        if len(trainer.optimizers) != 1:\n            raise MisconfigurationException(\"SWA currently works with 1 `optimizer`.\")\n\n        if len(trainer.lr_scheduler_configs) > 1:\n            raise MisconfigurationException(\"SWA currently not supported for more than 1 `lr_scheduler`.\")\n\n        assert trainer.max_epochs is not None\n        if isinstance(self._swa_epoch_start, float):\n            if trainer.max_epochs == -1:\n                raise MisconfigurationException(\n                    \"SWA with `swa_epoch_start` as a float is not supported when `max_epochs=-1`. \"\n                    \"Please provide `swa_epoch_start` as an integer.\"\n                )\n            self._swa_epoch_start = int(trainer.max_epochs * self._swa_epoch_start)\n\n        self._model_contains_batch_norm = self.pl_module_contains_batch_norm(pl_module)\n\n        self._max_epochs = trainer.max_epochs\n        if self._model_contains_batch_norm and trainer.max_epochs != -1:\n            # virtually increase max_epochs to perform batch norm update on latest epoch.","sourceCodeStart":143,"sourceCodeEnd":179,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/callbacks/stochastic_weight_avg.py#L143-L179","documentation":"At on_fit_start, StochasticWeightAveraging verifies the trainer has exactly one optimizer because its averaging and LR-constant logic only handles a single optimizer. If configure_optimizers returned more than one, this MisconfigurationException is raised.","triggerScenarios":"configure_optimizers returns a list/tuple of 2+ optimizers (multiple models or param groups per optimizer is fine, multiple optimizer objects is not) while SWA is in callbacks.","commonSituations":"Using separate optimizers for generator/discriminator, or per-module optimizers, and adding SWA for better checkpoints.","solutions":["Consolidate into one optimizer with multiple param groups: torch.optim.Adam([{ 'params': a.parameters()}, {'params': b.parameters(), 'lr': 1e-4}])","Or remove SWA when multiple optimizers are genuinely required"],"exampleFix":"# before\ndef configure_optimizers(self):\n    return [torch.optim.Adam(self.enc.parameters()), torch.optim.Adam(self.dec.parameters())]\n# after\ndef configure_optimizers(self):\n    opt = torch.optim.Adam([\n        {\"params\": self.enc.parameters()},\n        {\"params\": self.dec.parameters(), \"lr\": 1e-4},\n    ])\n    return opt","handlingStrategy":"validation","validationCode":"class MyModule(LightningModule):\n    def configure_optimizers(self):\n        if getattr(self, '_n_optimizers', 1) > 1 and self.use_swa:\n            raise ValueError('SWA requires one optimizer; merge param groups')\n        return torch.optim.Adam([\n            {\"params\": self.enc.parameters()},\n            {\"params\": self.dec.parameters(), \"lr\": 1e-4},\n        ])","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Model multiple param groups via one optimizer with group dicts","Conditionally add SWA only when len(trainer.optimizers)==1 is guaranteed"],"tags":["swa","multiple-optimizers","configure-optimizers","callback"],"backgroundTag":"callback-optimizer-count-mismatch","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}