Lightning-AI/pytorch-lightning · error · ValueError

When using multiple models + deepspeed, please provide the m

Error message

When using multiple models + deepspeed, please provide the model used to perform the optimization: `self.backward(loss, model=model)`

What it means

With DeepSpeed, Fabric.backward(loss) without an explicit model needs to infer which DeepSpeedEngine to use. If more than one model was set up (self._models_setup > 1), the choice is ambiguous, so Fabric demands the target model be passed via the `model=` keyword.

Source

Thrown at src/lightning/fabric/fabric.py:512

            model as argument here.

        Example::

            loss = criterion(output, target)
            fabric.backward(loss)

            # With DeepSpeed and multiple models
            fabric.backward(loss, model=model)

        """
        module = model._forward_module if model is not None else model
        module, _ = _unwrap_compiled(module)
        if isinstance(self._strategy, DeepSpeedStrategy):
            if model is None:
                if self._models_setup == 0:
                    raise RuntimeError("No models were set up for backward. Did you forget to call `fabric.setup()`?")
                if self._models_setup > 1:
                    raise ValueError(
                        "When using multiple models + deepspeed, please provide the model used to perform"
                        " the optimization: `self.backward(loss, model=model)`"
                    )
                module = self._strategy.model
            else:
                # requires to attach the current `DeepSpeedEngine` for the `_FabricOptimizer.step` call.
                self._strategy._deepspeed_engine = module

        lightning.fabric.wrappers._in_fabric_backward = True
        try:
            self._strategy.backward(tensor, module, *args, **kwargs)
        finally:
            lightning.fabric.wrappers._in_fabric_backward = False

    def clip_gradients(
        self,
        module: Union[torch.nn.Module, _FabricModule],
        optimizer: Union[Optimizer, _FabricOptimizer],

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Pass the model whose loss you are backpropagating: fabric.backward(loss, model=model2)
2. Consolidate into a single nn.Module (e.g. a container holding both networks) and set that up once

Example fix

# before
fabric.backward(loss)  # two models set up with deepspeed
# after
fabric.backward(loss, model=model2)

if fabric._models_setup > 1:
    fabric.backward(loss, model=target_model)
else:
    fabric.backward(loss)

When it happens

Trigger: Calling fabric.setup()/setup_module() on two or more models with DeepSpeedStrategy and then calling fabric.backward(loss) without model=. Typical in multi-model setups (e.g. GAN, teacher-student, policy+critic) under deepspeed.

Common situations: Training adversarial or multi-network architectures with Fabric + DeepSpeed; splitting a pipeline into several wrapped modules.

Related errors

• No models were set up for backward. Did you forget to call `
• The `{type(self._strategy).__name__}` requires the model and
• Received multiple values for {', '.join(duplicated_plugin_ke
• Received both `precision={precision_input}` and `plugins={se
• accelerator set through both strategy class and accelerator

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/814018cfcd445ff0. Report an issue: GitHub.