Lightning-AI/pytorch-lightning · error · NotImplementedError

The Finetuning callback does not support running with the De

Error message

The Finetuning callback does not support running with the DeepSpeed strategy. Choose a different strategy or disable the callback.

What it means

The `BaseFinetuning` freeze/unfreeze callback manipulates optimizer param groups in ways that conflict with DeepSpeed's own parameter partitioning, so Lightning explicitly raises NotImplementedError in `setup()` when the strategy is DeepSpeedStrategy. The two cannot be combined.

Source

Thrown at src/lightning/pytorch/callbacks/finetuning.py:290

            train_bn: Whether to train the BatchNormalization layers.

        """
        BaseFinetuning.make_trainable(modules)
        params_lr = optimizer.param_groups[0]["lr"] if lr is None else float(lr)
        denom_lr = initial_denom_lr if lr is None else 1.0
        params = BaseFinetuning.filter_params(modules, train_bn=train_bn, requires_grad=True)
        params = BaseFinetuning.filter_on_optimizer(optimizer, params)
        if params:
            optimizer.add_param_group({"params": params, "lr": params_lr / denom_lr})

    @override
    def setup(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule", stage: str) -> None:
        self.freeze_before_training(pl_module)

        from lightning.pytorch.strategies import DeepSpeedStrategy

        if isinstance(trainer.strategy, DeepSpeedStrategy):
            raise NotImplementedError(
                "The Finetuning callback does not support running with the DeepSpeed strategy."
                " Choose a different strategy or disable the callback."
            )

    @staticmethod
    def _apply_mapping_to_param_groups(param_groups: list[dict[str, Any]], mapping: dict) -> list[dict[str, Any]]:
        output = []
        for g in param_groups:
            # skip params to save memory
            group_state = {k: v for k, v in g.items() if k != "params"}
            group_state["params"] = [mapping[p] for p in g["params"]]
            output.append(group_state)
        return output

    def _store(
        self,
        pl_module: "pl.LightningModule",
        opt_idx: int,

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Remove the finetuning callback and rely on DeepSpeed's ZeRO + your own freeze logic in `configure_optimizers`/`setup`
2. Implement freezing manually via `module.requires_grad_(False)` in `on_fit_start` or the module's `setup` hook instead of the callback
3. Switch to a non-DeepSpeed strategy (e.g. FSDP or DDP) if the finetuning callback is essential

Example fix

# before
Trainer(strategy='deepspeed', callbacks=[BackboneFinetuning(...)])
# after
Trainer(strategy='deepspeed')  # freeze manually:
# model.backbone.requires_grad_(False) in LightningModule.setup(self, stage=None)

from lightning.pytorch.strategies import DeepSpeedStrategy
if isinstance(trainer.strategy, DeepSpeedStrategy):
    callbacks = [c for c in callbacks if not isinstance(c, BaseFinetuning)]

def finetuning_supported(trainer) -> bool:
    from lightning.pytorch.strategies import DeepSpeedStrategy
    return not isinstance(trainer.strategy, DeepSpeedStrategy)

When it happens

Trigger: Passing both `callbacks=[BaseFinetuningsubclass(...)]` and `strategy='deepspeed'` (or a DeepSpeedStrategy instance) to the Trainer. Raised as soon as `setup()` runs at the start of fitting.

Common situations: Porting a finetuning recipe (e.g. BackboneFinetuning) to a DeepSpeed config for large models; enabling DeepSpeed stage 2/3 for memory savings while keeping the existing callback list.

Related errors

• The `{type(trainer.strategy).__name__}` does not support `ac
• No models were set up for backward. Did you forget to call `
• When using multiple models + deepspeed, please provide the m
• The `{type(self._strategy).__name__}` requires the model and
• The LightningModule should have a nn.Module `backbone` attri

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/27cdd902b09e6166. Report an issue: GitHub.