{"record":{"id":"27cdd902b09e6166","repo":"Lightning-AI/pytorch-lightning","slug":"the-finetuning-callback-does-not-support-running-w","errorCode":null,"errorMessage":"The Finetuning callback does not support running with the DeepSpeed strategy. Choose a different strategy or disable the callback.","messagePattern":"The Finetuning callback does not support running with the DeepSpeed strategy\\. Choose a different strategy or disable the callback\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/callbacks/finetuning.py","lineNumber":290,"sourceCode":"            train_bn: Whether to train the BatchNormalization layers.\n\n        \"\"\"\n        BaseFinetuning.make_trainable(modules)\n        params_lr = optimizer.param_groups[0][\"lr\"] if lr is None else float(lr)\n        denom_lr = initial_denom_lr if lr is None else 1.0\n        params = BaseFinetuning.filter_params(modules, train_bn=train_bn, requires_grad=True)\n        params = BaseFinetuning.filter_on_optimizer(optimizer, params)\n        if params:\n            optimizer.add_param_group({\"params\": params, \"lr\": params_lr / denom_lr})\n\n    @override\n    def setup(self, trainer: \"pl.Trainer\", pl_module: \"pl.LightningModule\", stage: str) -> None:\n        self.freeze_before_training(pl_module)\n\n        from lightning.pytorch.strategies import DeepSpeedStrategy\n\n        if isinstance(trainer.strategy, DeepSpeedStrategy):\n            raise NotImplementedError(\n                \"The Finetuning callback does not support running with the DeepSpeed strategy.\"\n                \" Choose a different strategy or disable the callback.\"\n            )\n\n    @staticmethod\n    def _apply_mapping_to_param_groups(param_groups: list[dict[str, Any]], mapping: dict) -> list[dict[str, Any]]:\n        output = []\n        for g in param_groups:\n            # skip params to save memory\n            group_state = {k: v for k, v in g.items() if k != \"params\"}\n            group_state[\"params\"] = [mapping[p] for p in g[\"params\"]]\n            output.append(group_state)\n        return output\n\n    def _store(\n        self,\n        pl_module: \"pl.LightningModule\",\n        opt_idx: int,","sourceCodeStart":272,"sourceCodeEnd":308,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/callbacks/finetuning.py#L272-L308","documentation":"The `BaseFinetuning` freeze/unfreeze callback manipulates optimizer param groups in ways that conflict with DeepSpeed's own parameter partitioning, so Lightning explicitly raises NotImplementedError in `setup()` when the strategy is DeepSpeedStrategy. The two cannot be combined.","triggerScenarios":"Passing both `callbacks=[BaseFinetuningsubclass(...)]` and `strategy='deepspeed'` (or a DeepSpeedStrategy instance) to the Trainer. Raised as soon as `setup()` runs at the start of fitting.","commonSituations":"Porting a finetuning recipe (e.g. BackboneFinetuning) to a DeepSpeed config for large models; enabling DeepSpeed stage 2/3 for memory savings while keeping the existing callback list.","solutions":["Remove the finetuning callback and rely on DeepSpeed's ZeRO + your own freeze logic in `configure_optimizers`/`setup`","Implement freezing manually via `module.requires_grad_(False)` in `on_fit_start` or the module's `setup` hook instead of the callback","Switch to a non-DeepSpeed strategy (e.g. FSDP or DDP) if the finetuning callback is essential"],"exampleFix":"# before\nTrainer(strategy='deepspeed', callbacks=[BackboneFinetuning(...)])\n# after\nTrainer(strategy='deepspeed')  # freeze manually:\n# model.backbone.requires_grad_(False) in LightningModule.setup(self, stage=None)","handlingStrategy":"validation","validationCode":"from lightning.pytorch.strategies import DeepSpeedStrategy\nif isinstance(trainer.strategy, DeepSpeedStrategy):\n    callbacks = [c for c in callbacks if not isinstance(c, BaseFinetuning)]","typeGuard":"def finetuning_supported(trainer) -> bool:\n    from lightning.pytorch.strategies import DeepSpeedStrategy\n    return not isinstance(trainer.strategy, DeepSpeedStrategy)","tryCatchPattern":null,"preventionTips":["Keep a compatibility matrix of strategy vs callbacks in project docs","Build callback lists conditioned on the strategy programmatically"],"tags":["lightning","finetuning","deepspeed","strategy-incompatible"],"backgroundTag":"strategy-callback-incompatible","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}