{"record":{"id":"fd8da1d9b7b0a09c","repo":"Lightning-AI/pytorch-lightning","slug":"deepspeed-currently-only-supports-single-optimizer","errorCode":null,"errorMessage":"DeepSpeed currently only supports single optimizer, single optional scheduler.","messagePattern":"DeepSpeed currently only supports single optimizer, single optional scheduler\\.","errorType":"exception","errorClass":"MisconfigurationException","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/strategies/deepspeed.py","lineNumber":483,"sourceCode":"                \" The hook will still be called. Consider setting\"\n                \" `Trainer(gradient_clip_val=..., gradient_clip_algorithm='norm')`\"\n                \" which will use the internal mechanism.\"\n            )\n\n        if self.lightning_module.trainer.gradient_clip_algorithm == GradClipAlgorithmType.VALUE:\n            raise MisconfigurationException(\"DeepSpeed does not support clipping gradients by value.\")\n\n        assert isinstance(self.model, pl.LightningModule)\n        if self.lightning_module.trainer and self.lightning_module.trainer.training:\n            self._initialize_deepspeed_train(self.model)\n        else:\n            self._initialize_deepspeed_inference(self.model)\n\n    def _init_optimizers(self) -> tuple[Optimizer, Optional[LRSchedulerConfig]]:\n        assert self.lightning_module is not None\n        optimizers, lr_schedulers = _init_optimizers_and_lr_schedulers(self.lightning_module)\n        if len(optimizers) > 1 or len(lr_schedulers) > 1:\n            raise MisconfigurationException(\n                \"DeepSpeed currently only supports single optimizer, single optional scheduler.\"\n            )\n        return optimizers[0], lr_schedulers[0] if lr_schedulers else None\n\n    @property\n    def zero_stage_3(self) -> bool:\n        assert isinstance(self.config, dict)\n        zero_optimization = self.config.get(\"zero_optimization\")\n        return zero_optimization is not None and zero_optimization.get(\"stage\") == 3\n\n    def _initialize_deepspeed_train(self, model: Module) -> None:\n        optimizer, scheduler = None, None\n        assert isinstance(self.config, dict)\n        if \"optimizer\" in self.config:\n            rank_zero_info(\n                \"You have specified an optimizer and/or scheduler within the DeepSpeed config.\"\n                \" It is recommended to define it in `LightningModule.configure_optimizers`.\"\n            )","sourceCodeStart":465,"sourceCodeEnd":501,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/strategies/deepspeed.py#L465-L501","documentation":"When DeepSpeed initializes training it calls the module's `configure_optimizers` via _init_optimizers_and_lr_schedulers and requires at most one optimizer and at most one LR scheduler; anything more raises MisconfigurationException because the DeepSpeed engine manages a single optimizer/scheduler pair.","triggerScenarios":"`configure_optimizers()` returning 2+ optimizers or 2+ LR schedulers (e.g. `return [opt], [sched1, sched2]`) when training with DeepSpeedStrategy.","commonSituations":"Modules with warmup+decay modeled as two schedulers, or multiple optimizers each with its own scheduler; code written for the default strategy reused under DeepSpeed.","solutions":["Return at most one optimizer and one scheduler: `return optimizer` or `return [optimizer], [scheduler]`","Combine schedulers with ChainScheduler / SequentialLR from torch into a single scheduler","Use a non-DeepSpeed strategy if the multi-scheduler setup is mandatory"],"exampleFix":"# before\ndef configure_optimizers(self):\n    opt = torch.optim.AdamW(self.parameters())\n    return [opt], [warmup_sched, cosine_sched]\n\n# after\nfrom torch.optim.lr_scheduler import SequentialLR\nsched = SequentialLR(opt, [warmup_sched, cosine_sched], milestones=[1000])\nreturn opt, sched","handlingStrategy":"validation","validationCode":"out = model.configure_optimizers()\nopts = out[0] if isinstance(out, (list, tuple)) and out and isinstance(out[0], (list, tuple)) else out\nscheds = out[1] if isinstance(out, (list, tuple)) and len(out) == 2 else None\nassert len(opts if isinstance(opts, list) else [opts]) <= 1\nassert not isinstance(scheds, list) or len(scheds) <= 1","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Return a single optimizer and at most one scheduler from configure_optimizers","Merge schedulers with SequentialLR/ChainLR instead of returning lists"],"tags":["deepspeed","lr-scheduler","configure-optimizers"],"backgroundTag":"unsupported-multiple-optimizers","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}