Lightning-AI/pytorch-lightning · error · MisconfigurationException

DeepSpeed does not support clipping gradients by value.

Error message

DeepSpeed does not support clipping gradients by value.

What it means

DeepSpeed clips gradients internally (via its config `gradient_clipping`) and only supports norm-based clipping; value-based clipping (`gradient_clip_algorithm='value'`) has no DeepSpeed equivalent, so Lightning raises MisconfigurationException at engine init.

Source

Thrown at src/lightning/pytorch/strategies/deepspeed.py:471

            lr_scheduler=lr_scheduler,
            dist_init_required=False,
        )
        return deepspeed_engine, deepspeed_optimizer

    def init_deepspeed(self) -> None:
        assert self.lightning_module is not None
        # deepspeed handles gradient clipping internally
        if is_overridden("configure_gradient_clipping", self.lightning_module, pl.LightningModule):
            rank_zero_warn(
                "Since DeepSpeed handles gradient clipping internally, the default"
                " `LightningModule.configure_gradient_clipping` implementation will not actually clip gradients."
                " The hook will still be called. Consider setting"
                " `Trainer(gradient_clip_val=..., gradient_clip_algorithm='norm')`"
                " which will use the internal mechanism."
            )

        if self.lightning_module.trainer.gradient_clip_algorithm == GradClipAlgorithmType.VALUE:
            raise MisconfigurationException("DeepSpeed does not support clipping gradients by value.")

        assert isinstance(self.model, pl.LightningModule)
        if self.lightning_module.trainer and self.lightning_module.trainer.training:
            self._initialize_deepspeed_train(self.model)
        else:
            self._initialize_deepspeed_inference(self.model)

    def _init_optimizers(self) -> tuple[Optimizer, Optional[LRSchedulerConfig]]:
        assert self.lightning_module is not None
        optimizers, lr_schedulers = _init_optimizers_and_lr_schedulers(self.lightning_module)
        if len(optimizers) > 1 or len(lr_schedulers) > 1:
            raise MisconfigurationException(
                "DeepSpeed currently only supports single optimizer, single optional scheduler."
            )
        return optimizers[0], lr_schedulers[0] if lr_schedulers else None

    @property
    def zero_stage_3(self) -> bool:

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Switch to `Trainer(..., gradient_clip_val=X, gradient_clip_algorithm="norm")`
2. Or drop the Trainer clipping arg and set `gradient_clipping` in the DeepSpeed config dict
3. Avoid `configure_gradient_clipping` implementations that rely on value clipping under DeepSpeed

Example fix

# before
trainer = Trainer(strategy=DeepSpeedStrategy(config=cfg), gradient_clip_val=1.0, gradient_clip_algorithm="value")

# after
trainer = Trainer(strategy=DeepSpeedStrategy(config=cfg), gradient_clip_val=1.0, gradient_clip_algorithm="norm")

algo = "norm"  # ensure value never reaches DeepSpeed
trainer = Trainer(strategy=DeepSpeedStrategy(config=cfg), gradient_clip_val=1.0, gradient_clip_algorithm=algo)

When it happens

Trigger: `Trainer(strategy=DeepSpeedStrategy(...), gradient_clip_val=X, gradient_clip_algorithm="value")`, or the equivalent LightningModule `configure_gradient_clipping` expecting value clipping — detected in `init_deepspeed` before the engine is built.

Common situations: Porting a recipe that used value clipping under DDP; default Trainer flags copied from another project. Fix: use `gradient_clip_algorithm="norm"` (or omit it) and optionally set gradient_clipping in the DeepSpeed config.

Related errors

• DeepSpeed handles gradient clipping automatically within the
• No models were set up for backward. Did you forget to call `
• When using multiple models + deepspeed, please provide the m
• Only one of `clip_val` or `max_norm` can be set as this spec
• You have to specify either `clip_val` or `max_norm` to do gr

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/6b63a37d25a2d1fc. Report an issue: GitHub.