{"record":{"id":"c3ee1b4010b3c3a0","repo":"Lightning-AI/pytorch-lightning","slug":"the-current-optimizer-type-optimizer-qualname","errorCode":null,"errorMessage":"The current optimizer, {type(optimizer).__qualname__}, does not allow for gradient clipping because it performs unscaling of gradients internally. HINT: Are you using a 'fused' optimizer?","messagePattern":"The current optimizer, (.+?), does not allow for gradient clipping because it performs unscaling of gradients internally\\. HINT: Are you using a 'fused' optimizer\\?","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/plugins/precision/amp.py","lineNumber":127,"sourceCode":"        self._after_closure(model, optimizer)\n\n        # in manual optimization, the closure does not return a value\n        if not skip_unscaling:\n            # note: the scaler will skip the `optimizer.step` if nonfinite gradients are found\n            step_output = self.scaler.step(optimizer, **kwargs)  # type: ignore[arg-type]\n            self.scaler.update()\n            return step_output\n        return closure_result\n\n    @override\n    def clip_gradients(\n        self,\n        optimizer: Optimizer,\n        clip_val: Union[int, float] = 0.0,\n        gradient_clip_algorithm: GradClipAlgorithmType = GradClipAlgorithmType.NORM,\n    ) -> None:\n        if clip_val > 0 and self.scaler is not None and _optimizer_handles_unscaling(optimizer):\n            raise RuntimeError(\n                f\"The current optimizer, {type(optimizer).__qualname__}, does not allow for gradient clipping\"\n                \" because it performs unscaling of gradients internally. HINT: Are you using a 'fused' optimizer?\"\n            )\n        super().clip_gradients(optimizer=optimizer, clip_val=clip_val, gradient_clip_algorithm=gradient_clip_algorithm)\n\n    def autocast_context_manager(self) -> torch.autocast:\n        dtype = torch.bfloat16 if self.precision == \"bf16-mixed\" else torch.half\n        return torch.autocast(self.device, dtype=dtype, cache_enabled=False)\n\n    @override\n    @contextmanager\n    def forward_context(self) -> Generator[None, None, None]:\n        \"\"\"Enable autocast and clear cached casts after nested grad-disabling contexts exit.\"\"\"\n        original_no_grad = torch.no_grad\n        original_inference_mode = torch.inference_mode\n\n        def _clear_cache_on_exit(\n            context_factory: Callable[..., Any], *, clear_cache: Callable[..., bool]","sourceCodeStart":109,"sourceCodeEnd":145,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/plugins/precision/amp.py#L109-L145","documentation":"Gradient clipping was requested (clip_val > 0) with an active GradScaler, but the optimizer performs its own unsscaling internally (detected via _optimizer_handles_unscaling, e.g. fused optimizers like fused AdamW). Since the AMP plugin clips gradients after unscaling them with its own scaler, double-unsscaling would corrupt the gradients, so the combination is rejected.","triggerScenarios":"Trainer(precision='16-mixed', gradient_clip_val=1.0) with optimizer = torch.optim.AdamW(..., fused=True) (or any optimizer whose step handles unscaling); the plugin's clip_gradients is then called with a scaler present.","commonSituations":"Enabling fused=True for performance on newer GPUs while keeping AMP fp16 and gradient clipping; upgrading PyTorch where fused AdamW became common; following performance-tuning guides that recommend fused optimizers.","solutions":["Drop fused=True from the optimizer so the GradScaler handles unscaling and clipping","Keep the fused optimizer but switch precision to 'bf16-mixed' (no scaler, clipping allowed)","Disable gradient clipping (set Trainer(gradient_clip_val=0) or leave it None)"],"exampleFix":"# before\noptimizer = torch.optim.AdamW(self.parameters(), lr=1e-3, fused=True)\nTrainer(precision='16-mixed', gradient_clip_val=1.0)\n\n# after\noptimizer = torch.optim.AdamW(self.parameters(), lr=1e-3)\nTrainer(precision='16-mixed', gradient_clip_val=1.0)","handlingStrategy":"validation","validationCode":"def optimizer_allows_amp_clipping(optimizer) -> bool:\n    # fused optimizers unscale internally and cannot be clipped by the GradScaler\n    return not getattr(optimizer, 'fused', False)\n\nif trainer.gradient_clip_val and trainer.precision == '16-mixed':\n    assert optimizer_allows_amp_clipping(optimizer)","typeGuard":"def is_fused_optimizer(opt) -> bool:\n    return bool(getattr(opt, 'fused', False)) or type(opt).__name__ in {'FusedAdam','FusedSGD'}","tryCatchPattern":"try:\n    trainer.fit(model)\nexcept RuntimeError as e:\n    if 'does not allow for gradient clipping' in str(e):\n        optimizer = rebuild_without_fused(optimizer); trainer.fit(model)\n    else:\n        raise","preventionTips":["When enabling fused=True, simultaneously switch to bf16-mixed or drop gradient_clip_val","Encode the fused+clip+16-mixed conflict in your config linter"],"tags":["pytorch-lightning","amp","fused-optimizer","gradient-clipping","adamw","gradscaler"],"backgroundTag":"gradient-clipping-unscale-conflict","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}