{"record":{"id":"eda450983f6a4b1e","repo":"Lightning-AI/pytorch-lightning","slug":"gradient-clip-algorithm-norm-is-currently-not","errorCode":null,"errorMessage":"`gradient_clip_algorithm='norm'` is currently not supported for `{self.__class__.__name__}`","messagePattern":"`gradient_clip_algorithm='norm'` is currently not supported for `(.+?)`","errorType":"exception","errorClass":"MisconfigurationException","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/plugins/precision/fsdp.py","lineNumber":90,"sourceCode":"            \"bf16-true\": torch.bfloat16,\n            \"16-true\": torch.float16,\n            \"32-true\": torch.float32,\n        }\n        self._desired_input_dtype = precision_to_type[self.precision]\n\n    @override\n    def convert_module(self, module: Module) -> Module:\n        if \"true\" in self.precision:\n            return module.to(dtype=self._desired_input_dtype)\n        return module\n\n    @override\n    def clip_grad_by_norm(self, *_: Any, **__: Any) -> None:\n        # see https://pytorch.org/docs/stable/fsdp.html#torch.distributed.fsdp.FullyShardedDataParallel.clip_grad_norm_\n        # section `Gradient Clipping`, using `torch.nn.utils.clip_grad_norm_` is incorrect with FSDP.\n        # To overcome this we need to call root_sharded_module.clip_grad_norm(clip_val), but we don't have a reference\n        # to the root module\n        raise MisconfigurationException(\n            f\"`gradient_clip_algorithm='norm'` is currently not supported for `{self.__class__.__name__}`\"\n        )\n\n    @property\n    def mixed_precision_config(self) -> \"TorchMixedPrecision\":\n        from torch.distributed.fsdp.fully_sharded_data_parallel import MixedPrecision as TorchMixedPrecision\n\n        if self.precision in (\"16-true\", \"bf16-true\"):\n            rank_zero_warn(\n                f\"FSDP with `{self.precision}` enables computation in lower precision. \"\n                \"FSDP will always retain a full-precision copy of the model parameters for sharding.\"\n            )\n\n        if self.precision in (\"16-true\", \"16-mixed\"):\n            param_dtype = reduce_dtype = buffer_dtype = torch.float16\n        elif self.precision in (\"bf16-true\", \"bf16-mixed\"):\n            param_dtype = reduce_dtype = buffer_dtype = torch.bfloat16\n        elif self.precision == \"32-true\":","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/plugins/precision/fsdp.py#L72-L108","documentation":"FSDPMixedPrecisionPlugin.clip_grad_by_norm unconditionally raises because torch.nn.utils.clip_grad_norm_ is mathematically wrong for sharded FSDP parameters (norms must be reduced across shards). The correct API is the root module's clip_grad_norm, but the plugin has no reference to the root module, so Lightning blocks norm-based clipping for this plugin class instead of silently clipping incorrectly.","triggerScenarios":"Trainer(strategy='fsdp', gradient_clip_val=0.5, gradient_clip_algorithm='norm') (the default algorithm) with the FSDP mixed-precision plugin; the base clip_gradients dispatches to clip_grad_by_norm which raises.","commonSituations":"Copying gradient_clip_val from a DDP config into an FSDP run; default Trainer gradient_clip_algorithm being NORM so simply setting gradient_clip_val triggers it; FSDP1-based strategies in Lightning (FSDPStrategy/fully sharded plugins).","solutions":["Switch to value-based clipping: Trainer(..., gradient_clip_algorithm='value')","Use FSDP's own clipping via strategy configuration where supported (FSDP2 / newer Lightning exposes it)","Disable gradient clipping for the FSDP run"],"exampleFix":"# before\nTrainer(strategy='fsdp', gradient_clip_val=0.5)  # defaults to algorithm='norm'\n\n# after\nfrom lightning.pytorch.callbacks import GradientClipAlgorithmType  # or string\nTrainer(strategy='fsdp', gradient_clip_val=0.5, gradient_clip_algorithm='value')","handlingStrategy":"validation","validationCode":"from lightning.pytorch.trainer import Trainer\n\ndef clipped_fsdp_config(clip_val):\n    return dict(strategy='fsdp', gradient_clip_val=clip_val,\n                gradient_clip_algorithm='value')\n\ntrainer = Trainer(**clipped_fsdp_config(0.5))","typeGuard":"def clip_algorithm_supported_for_fsdp(alg: str) -> bool:\n    return alg == 'value'","tryCatchPattern":null,"preventionTips":["When migrating DDP configs to FSDP, always set gradient_clip_algorithm='value' if clipping","Document in the run config why norm clipping is forbidden under FSDP sharding"],"tags":["pytorch-lightning","fsdp","gradient-clipping","clip-norm","distributed"],"backgroundTag":"unsupported-gradient-clipping-mode","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}