{"record":{"id":"7861b598d6aa964c","repo":"Lightning-AI/pytorch-lightning","slug":"currently-only-one-optimizer-is-supported-with-dee-7861b5","errorCode":null,"errorMessage":"Currently only one optimizer is supported with DeepSpeed. Got {len(optimizers)} optimizers instead.","messagePattern":"Currently only one optimizer is supported with DeepSpeed\\. Got (.+?) optimizers instead\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/strategies/deepspeed.py","lineNumber":420,"sourceCode":"    @override\n    def restore_checkpoint_after_setup(self) -> bool:\n        return True\n\n    @override\n    def _setup_model_and_optimizers(\n        self, model: Module, optimizers: list[Optimizer]\n    ) -> tuple[\"deepspeed.DeepSpeedEngine\", list[Optimizer]]:\n        \"\"\"Setup a model and multiple optimizers together.\n\n        Currently only a single optimizer is supported.\n\n        Return:\n            The model wrapped into a :class:`deepspeed.DeepSpeedEngine` and a list with a single\n            deepspeed optimizer.\n\n        \"\"\"\n        if len(optimizers) != 1:\n            raise ValueError(\n                f\"Currently only one optimizer is supported with DeepSpeed. Got {len(optimizers)} optimizers instead.\"\n            )\n\n        # train_micro_batch_size_per_gpu is used for throughput logging purposes\n        # normally we set this to the batch size, but it is not available here unless the user provides it\n        # as part of the config\n        assert self.config is not None\n        self.config.setdefault(\"train_micro_batch_size_per_gpu\", 1)\n        self.model, optimizer = self._setup_model_and_optimizer(model, optimizers[0])\n        self._set_deepspeed_activation_checkpointing()\n        return self.model, [optimizer]\n\n    def _setup_model_and_optimizer(\n        self,\n        model: Module,\n        optimizer: Optional[Optimizer],\n        lr_scheduler: Optional[Union[LRScheduler, ReduceLROnPlateau]] = None,\n    ) -> tuple[\"deepspeed.DeepSpeedEngine\", Optimizer]:","sourceCodeStart":402,"sourceCodeEnd":438,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/strategies/deepspeed.py#L402-L438","documentation":"DeepSpeed (as integrated by Lightning) wraps exactly one optimizer into its engine, so _setup_model_and_optimizers rejects a list whose length differs from 1. `configure_optimizers` returning multiple optimizers is therefore unsupported under this strategy.","triggerScenarios":"A LightningModule whose `configure_optimizers()` returns a list/tuple of 2+ optimizers (or a dict with multiple optimizers) while training with `DeepSpeedStrategy`.","commonSituations":"Reusing a GAN / multi-model LightningModule (separate optimizers for generator and discriminator) with DeepSpeed; migrating a manual training loop that stepped several optimizers.","solutions":["Refactor `configure_optimizers` to return a single optimizer (e.g. use one optimizer over all parameters with param groups)","If multiple optimizers are essential, use a different strategy such as DDP","Check for accidental list-wrapping, e.g. `return [opt1, opt2]` where one optimizer suffices"],"exampleFix":"# before\ndef configure_optimizers(self):\n    return [torch.optim.AdamW(self.g.parameters()), torch.optim.AdamW(self.d.parameters())]\n\n# after\ndef configure_optimizers(self):\n    return torch.optim.AdamW(itertools.chain(self.g.parameters(), self.d.parameters()))","handlingStrategy":"validation","validationCode":"n = len(self.lr_scheduler_configs) if hasattr(self, \"lr_scheduler_configs\") else 0\noptimizers = self.configure_optimizers()  # in tests, assert shape before training\nassert not isinstance(optimizers, (list, tuple)) or len(optimizers) == 1, \"DeepSpeed supports one optimizer\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep single-optimizer LightningModules when targeting DeepSpeed","Unit-test configure_optimizers output shape when adding strategies"],"tags":["deepspeed","optimizers","multiple-optimizers","configure-optimizers"],"backgroundTag":"unsupported-multiple-optimizers","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}