Lightning-AI/pytorch-lightning · warning

You have overridden the `LightningModule.backward` hook but

Error message

You have overridden the `LightningModule.backward` hook but it will be ignored since DeepSpeed handles the backward logic internally.

What it means

Warning (not exception) emitted by DeepSpeedPrecision's backward(): if the LightningModule overrides the backward hook, DeepSpeed will ignore it because the DeepSpeed engine performs its own backward pass with its internal loss scaling.

Source

Thrown at src/lightning/pytorch/plugins/precision/deepspeed.py:112

        self,
        tensor: Tensor,
        model: "pl.LightningModule",
        optimizer: Optional[Steppable],
        *args: Any,
        **kwargs: Any,
    ) -> None:
        r"""Performs back-propagation using DeepSpeed's engine.

        Args:
            tensor: the loss tensor
            model: the model to be optimized
            optimizer: ignored for DeepSpeed
            \*args: additional positional arguments for the :meth:`deepspeed.DeepSpeedEngine.backward` call
            \**kwargs: additional keyword arguments for the :meth:`deepspeed.DeepSpeedEngine.backward` call

        """
        if is_overridden("backward", model):
            warning_cache.warn(
                "You have overridden the `LightningModule.backward` hook but it will be ignored since DeepSpeed handles"
                " the backward logic internally."
            )
        deepspeed_engine: deepspeed.DeepSpeedEngine = model.trainer.model
        deepspeed_engine.backward(tensor, *args, **kwargs)

    @override
    def optimizer_step(  # type: ignore[override]
        self,
        optimizer: Steppable,
        model: "pl.LightningModule",
        closure: Callable[[], Any],
        **kwargs: Any,
    ) -> Any:
        if isinstance(optimizer, LBFGS):
            raise MisconfigurationException("DeepSpeed and the LBFGS optimizer are not compatible.")
        closure_result = closure()
        self._after_closure(model, optimizer)

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Remove the backward override when using DeepSpeed; rely on DeepSpeed's engine.backward()
2. Move custom loss scaling into training_step or a precision plugin
3. If you need custom backward logic, switch to a non-DeepSpeed strategy

Example fix

# before
class M(LLightningModule):
    def backward(self, loss):
        loss.backward()
# after (with deepspeed strategy)
class M(LightningModule):
    # no backward override; DeepSpeed handles it
    pass

from lightning.pytorch.utilities.model_helpers import is_overridden
assert not is_overridden('backward', model), 'remove backward override for deepspeed'

When it happens

Trigger: Using Trainer(strategy='deepspeed', ...) with a LightningModule that defines def backward(self, loss): ...

Common situations: Porting a mixed-precision training script that manually calls loss.backward() or scales gradients in backward() to DeepSpeed.

Related errors

• Received both `precision={precision_input}` and `plugins={se
• precision set through both strategy class and plugins, choos
• Precision {repr(precision)} is invalid. Allowed precision va
• No models were set up for backward. Did you forget to call `
• When using multiple models + deepspeed, please provide the m

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/ba1a91af08dbd85b. Report an issue: GitHub.