{"record":{"id":"b7b1e5da1886ee2a","repo":"Lightning-AI/pytorch-lightning","slug":"swa-currently-not-supported-for-more-than-1-lr-sc","errorCode":null,"errorMessage":"SWA currently not supported for more than 1 `lr_scheduler`.","messagePattern":"SWA currently not supported for more than 1 `lr_scheduler`\\.","errorType":"exception","errorClass":"MisconfigurationException","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/callbacks/stochastic_weight_avg.py","lineNumber":164,"sourceCode":"    @staticmethod\n    def pl_module_contains_batch_norm(pl_module: \"pl.LightningModule\") -> bool:\n        return any(isinstance(module, nn.modules.batchnorm._BatchNorm) for module in pl_module.modules())\n\n    @override\n    def setup(self, trainer: \"pl.Trainer\", pl_module: \"pl.LightningModule\", stage: str) -> None:\n        if isinstance(trainer.strategy, (FSDPStrategy, DeepSpeedStrategy)):\n            raise MisconfigurationException(\"SWA does not currently support sharded models.\")\n\n        # copy the model before moving it to accelerator device.\n        self._average_model = deepcopy(pl_module)\n\n    @override\n    def on_fit_start(self, trainer: \"pl.Trainer\", pl_module: \"pl.LightningModule\") -> None:\n        if len(trainer.optimizers) != 1:\n            raise MisconfigurationException(\"SWA currently works with 1 `optimizer`.\")\n\n        if len(trainer.lr_scheduler_configs) > 1:\n            raise MisconfigurationException(\"SWA currently not supported for more than 1 `lr_scheduler`.\")\n\n        assert trainer.max_epochs is not None\n        if isinstance(self._swa_epoch_start, float):\n            if trainer.max_epochs == -1:\n                raise MisconfigurationException(\n                    \"SWA with `swa_epoch_start` as a float is not supported when `max_epochs=-1`. \"\n                    \"Please provide `swa_epoch_start` as an integer.\"\n                )\n            self._swa_epoch_start = int(trainer.max_epochs * self._swa_epoch_start)\n\n        self._model_contains_batch_norm = self.pl_module_contains_batch_norm(pl_module)\n\n        self._max_epochs = trainer.max_epochs\n        if self._model_contains_batch_norm and trainer.max_epochs != -1:\n            # virtually increase max_epochs to perform batch norm update on latest epoch.\n            assert trainer.fit_loop.max_epochs is not None\n            trainer.fit_loop.max_epochs += 1\n","sourceCodeStart":146,"sourceCodeEnd":182,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/callbacks/stochastic_weight_avg.py#L146-L182","documentation":"StochasticWeightAveraging supports at most one learning-rate scheduler (it replaces scheduling with a constant/annealed SWA LR after swa_epoch_start). At on_fit_start, more than one lr_scheduler config triggers this MisconfigurationException.","triggerScenarios":"configure_optimizers returns one optimizer with a list of 2+ LRScheduler/LightningModule hyperparameter dict scheduler entries, e.g. [ReduceLROnPlateau, StepLR], with SWA enabled.","commonSituations":"Combining warmup + decay as two separate schedulers instead of chained/sequential schedulers; adding SWA to an existing multi-scheduler setup.","solutions":["Merge schedulers into one (e.g. use SequentialLR or a single cosine schedule with warmup)","Or keep only the primary scheduler","Or remove the SWA callback if multiple schedulers are required"],"exampleFix":"# before\ndef configure_optimizers(self):\n    opt = torch.optim.AdamW(self.parameters())\n    return [opt], [torch.optim.lr_scheduler.StepLR(opt, 10), torch.optim.lr_scheduler.ReduceLROnPlateau(opt)]\n# after\nfrom torch.optim.lr_scheduler import SequentialLR\nopt = torch.optim.AdamW(self.parameters())\nsched = SequentialLR(opt, [WarmupLR(opt, 5), CosineAnnealingLR(opt, 95)], milestones=[5])\nreturn [opt], [sched]","handlingStrategy":"validation","validationCode":"from torch.optim.lr_scheduler import SequentialLR\ndef configure_optimizers(self):\n    opt = torch.optim.AdamW(self.parameters(), lr=1e-3)\n    sched = SequentialLR(opt, self.sched_list, milestones=self.milestones)\n    return [opt], [sched]  # exactly one scheduler config","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Compose warmup+decay with SequentialLR instead of scheduler lists","Audit configure_optimizers return arity before enabling SWA"],"tags":["swa","lr-scheduler","multiple-schedulers","callback"],"backgroundTag":"callback-scheduler-count-mismatch","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}