{"record":{"id":"5e5b6bd346190b97","repo":"Lightning-AI/pytorch-lightning","slug":"currently-only-one-optimizer-is-supported-with-dee","errorCode":null,"errorMessage":"Currently only one optimizer is supported with DeepSpeed. Got {len(optimizers)} optimizers instead.","messagePattern":"Currently only one optimizer is supported with DeepSpeed\\. Got (.+?) optimizers instead\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/strategies/deepspeed.py","lineNumber":351,"sourceCode":"\n    @property\n    def model(self) -> \"DeepSpeedEngine\":\n        return self._deepspeed_engine\n\n    @override\n    def setup_module_and_optimizers(\n        self, module: Module, optimizers: list[Optimizer], scheduler: Optional[\"_LRScheduler\"] = None\n    ) -> tuple[\"DeepSpeedEngine\", list[Optimizer], Any]:\n        \"\"\"Set up a model and multiple optimizers together, along with an optional learning rate scheduler. Currently,\n        only a single optimizer is supported.\n\n        Return:\n            The model wrapped into a :class:`deepspeed.DeepSpeedEngine`, a list with a single\n            deepspeed optimizer, and an optional learning rate scheduler.\n\n        \"\"\"\n        if len(optimizers) != 1:\n            raise ValueError(\n                f\"Currently only one optimizer is supported with DeepSpeed. Got {len(optimizers)} optimizers instead.\"\n            )\n\n        self._deepspeed_engine, optimizer, scheduler = self._initialize_engine(module, optimizers[0], scheduler)\n        self._set_deepspeed_activation_checkpointing()\n        return self._deepspeed_engine, [optimizer], scheduler\n\n    @override\n    def setup_module(self, module: Module) -> \"DeepSpeedEngine\":\n        \"\"\"Set up a module for inference (no optimizers).\n\n        For training, see :meth:`setup_module_and_optimizers`.\n\n        \"\"\"\n        self._deepspeed_engine, _, _ = self._initialize_engine(module)\n        return self._deepspeed_engine\n\n    @override","sourceCodeStart":333,"sourceCodeEnd":369,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/strategies/deepspeed.py#L333-L369","documentation":"DeepSpeed's engine initialization takes exactly one optimizer. DeepSpeedStrategy.setup_module_and_optimizers raises ValueError if the optimizers list has any length other than 1, because the DeepSpeedEngine API cannot represent multiple optimizers.","triggerScenarios":"fabric.setup(module, optimizer1, optimizer2) or strategy.setup_module_and_optimizers(module, [opt1, opt2]) with more than one optimizer under DeepSpeedStrategy; also passing zero optimizers.","commonSituations":"GAN-style or multi-model training with separate optimizers; porting a DDP script that sets up several optimizers to DeepSpeed.","solutions":["Restructure training to a single optimizer (e.g. combine parameters into one optimizer, or train models in separate Fabric runs/checkpoints)","If you need multiple models, create one Fabric/DeepSpeedStrategy instance per model","Use a different strategy (DDP/FSDP) if multiple optimizers are a hard requirement"],"exampleFix":"# before\nmodel, (opt_g, opt_d) = fabric.setup(module, opt_g, opt_d)  # under deepspeed\n# after\nmodel, opt_g = fabric.setup(module, opt_g)\n# train discriminator in a separate run/stage","handlingStrategy":"validation","validationCode":"assert len(optimizers) == 1, \"DeepSpeed supports exactly one optimizer\"","typeGuard":"def deepspeed_optimizer_count_ok(optimizers) -> bool:\n    return len(list(optimizers)) == 1","tryCatchPattern":"try:\n    model, optimizers, scheduler = strategy.setup_module_and_optimizers(module, optimizers)\nexcept ValueError:\n    # split training into per-model runs","preventionTips":["Design single-optimizer training loops for DeepSpeed","Branch setup code on isinstance(strategy, DeepSpeedStrategy)"],"tags":["deepspeed","multiple-optimizers","unsupported-operation","pytorch-lightning"],"backgroundTag":"unsupported-operation-combination","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}