{"record":{"id":"dee335b15205efd0","repo":"Lightning-AI/pytorch-lightning","slug":"deepspeed-handles-gradient-clipping-automatically","errorCode":null,"errorMessage":"DeepSpeed handles gradient clipping automatically within the optimizer. Make sure to set the `gradient_clipping` value in your Config.","messagePattern":"DeepSpeed handles gradient clipping automatically within the optimizer\\. Make sure to set the `gradient_clipping` value in your Config\\.","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/strategies/deepspeed.py","lineNumber":563,"sourceCode":"                \" or a single checkpoint file by setting `DeepSpeedStrategy(..., load_full_weights=True)`.\"\n            )\n\n        # `Engine.load_checkpoint` adds useless keys 'optimizer' and 'lr_scheduler' to the client state; remove\n        # them to avoid name collision with user state\n        keys = set(client_state) & set(state) - {\"optimizer\", \"lr_scheduler\"}\n        _move_state_into(source=client_state, destination=state, keys=keys)\n        return client_state\n\n    @override\n    def clip_gradients_norm(\n        self,\n        module: \"DeepSpeedEngine\",\n        optimizer: Optimizer,\n        max_norm: Union[float, int],\n        norm_type: Union[float, int] = 2.0,\n        error_if_nonfinite: bool = True,\n    ) -> torch.Tensor:\n        raise NotImplementedError(\n            \"DeepSpeed handles gradient clipping automatically within the optimizer. \"\n            \"Make sure to set the `gradient_clipping` value in your Config.\"\n        )\n\n    @override\n    def clip_gradients_value(\n        self, module: \"DeepSpeedEngine\", optimizer: Optimizer, clip_val: Union[float, int]\n    ) -> None:\n        raise NotImplementedError(\n            \"DeepSpeed handles gradient clipping automatically within the optimizer. \"\n            \"Make sure to set the `gradient_clipping` value in your Config.\"\n        )\n\n    @classmethod\n    @override\n    def register_strategies(cls, strategy_registry: _StrategyRegistry) -> None:\n        strategy_registry.register(\"deepspeed\", cls, description=\"Default DeepSpeed Strategy\")\n        strategy_registry.register(\"deepspeed_stage_1\", cls, description=\"DeepSpeed with ZeRO Stage 1 enabled\", stage=1)","sourceCodeStart":545,"sourceCodeEnd":581,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/strategies/deepspeed.py#L545-L581","documentation":"DeepSpeed performs gradient clipping inside its engine/optimizer using the gradient_clipping entry of the DeepSpeed config. Lightning therefore explicitly does not implement clip_gradients_norm for this strategy and raises NotImplementedError if called.","triggerScenarios":"Calling strategy.clip_gradients_norm(...) (directly, or via Fabric's Trainer/autocast gradient-clip path that is not DeepSpeed-aware) on a DeepSpeedStrategy instance.","commonSituations":"User code or a custom loop calls fabric.clip_gradients... or the strategy method manually; a LightningModule sets Trainer(gradient_clip_val=...) without the corresponding gradient_clipping key in the DeepSpeed config; porting code from DDPStrategy to DeepSpeed.","solutions":["Set gradient clipping in the DeepSpeed config: config['gradient_clipping'] = <float> (add gradient_clipping to the JSON/dict)","Remove manual calls to clip_gradients_norm on the strategy and let the DeepSpeed engine clip during optimizer.step()","If using the Trainer, ensure Trainer(gradient_clip_val=...) matches the DeepSpeed config value"],"exampleFix":"# before\nstrategy.clip_gradients_norm(module, optimizer, max_norm=1.0)\n\n# after\nds_config = {\"gradient_clipping\": 1.0, ...}\nstrategy = DeepSpeedStrategy(config=ds_config)","handlingStrategy":"validation","validationCode":"ds_config.setdefault(\"gradient_clipping\", 1.0)\nassert \"gradient_clipping\" in ds_config, \"set gradient_clipping in DeepSpeed config\"","typeGuard":"def supports_norm_clip(strategy) -> bool:\n    return not type(strategy).__name__ == \"DeepSpeedStrategy\"","tryCatchPattern":null,"preventionTips":["Put gradient clipping in the DeepSpeed config, not the code","Never call strategy clip methods directly with DeepSpeed"],"tags":["deepspeed","gradient-clipping","not-implemented"],"backgroundTag":"gradient-clipping-misconfigured","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}