Lightning-AI/pytorch-lightning · error · MisconfigurationException

DeepSpeed currently only supports single optimizer, single o

Error message

DeepSpeed currently only supports single optimizer, single optional scheduler.

What it means

When DeepSpeed initializes training it calls the module's `configure_optimizers` via _init_optimizers_and_lr_schedulers and requires at most one optimizer and at most one LR scheduler; anything more raises MisconfigurationException because the DeepSpeed engine manages a single optimizer/scheduler pair.

Source

Thrown at src/lightning/pytorch/strategies/deepspeed.py:483

                " The hook will still be called. Consider setting"
                " `Trainer(gradient_clip_val=..., gradient_clip_algorithm='norm')`"
                " which will use the internal mechanism."
            )

        if self.lightning_module.trainer.gradient_clip_algorithm == GradClipAlgorithmType.VALUE:
            raise MisconfigurationException("DeepSpeed does not support clipping gradients by value.")

        assert isinstance(self.model, pl.LightningModule)
        if self.lightning_module.trainer and self.lightning_module.trainer.training:
            self._initialize_deepspeed_train(self.model)
        else:
            self._initialize_deepspeed_inference(self.model)

    def _init_optimizers(self) -> tuple[Optimizer, Optional[LRSchedulerConfig]]:
        assert self.lightning_module is not None
        optimizers, lr_schedulers = _init_optimizers_and_lr_schedulers(self.lightning_module)
        if len(optimizers) > 1 or len(lr_schedulers) > 1:
            raise MisconfigurationException(
                "DeepSpeed currently only supports single optimizer, single optional scheduler."
            )
        return optimizers[0], lr_schedulers[0] if lr_schedulers else None

    @property
    def zero_stage_3(self) -> bool:
        assert isinstance(self.config, dict)
        zero_optimization = self.config.get("zero_optimization")
        return zero_optimization is not None and zero_optimization.get("stage") == 3

    def _initialize_deepspeed_train(self, model: Module) -> None:
        optimizer, scheduler = None, None
        assert isinstance(self.config, dict)
        if "optimizer" in self.config:
            rank_zero_info(
                "You have specified an optimizer and/or scheduler within the DeepSpeed config."
                " It is recommended to define it in `LightningModule.configure_optimizers`."
            )

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Return at most one optimizer and one scheduler: `return optimizer` or `return [optimizer], [scheduler]`
2. Combine schedulers with ChainScheduler / SequentialLR from torch into a single scheduler
3. Use a non-DeepSpeed strategy if the multi-scheduler setup is mandatory

Example fix

# before
def configure_optimizers(self):
    opt = torch.optim.AdamW(self.parameters())
    return [opt], [warmup_sched, cosine_sched]

# after
from torch.optim.lr_scheduler import SequentialLR
sched = SequentialLR(opt, [warmup_sched, cosine_sched], milestones=[1000])
return opt, sched

out = model.configure_optimizers()
opts = out[0] if isinstance(out, (list, tuple)) and out and isinstance(out[0], (list, tuple)) else out
scheds = out[1] if isinstance(out, (list, tuple)) and len(out) == 2 else None
assert len(opts if isinstance(opts, list) else [opts]) <= 1
assert not isinstance(scheds, list) or len(scheds) <= 1

When it happens

Trigger: `configure_optimizers()` returning 2+ optimizers or 2+ LR schedulers (e.g. `return [opt], [sched1, sched2]`) when training with DeepSpeedStrategy.

Common situations: Modules with warmup+decay modeled as two schedulers, or multiple optimizers each with its own scheduler; code written for the default strategy reused under DeepSpeed.

Related errors

• The lr scheduler dict must have the key "scheduler" with its
• Currently only one optimizer is supported with DeepSpeed. Go
• No models were set up for backward. Did you forget to call `
• When using multiple models + deepspeed, please provide the m
• The `{type(self._strategy).__name__}` requires the model and

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/fd8da1d9b7b0a09c. Report an issue: GitHub.