{"record":{"id":"7047705a10f68b3a","repo":"Lightning-AI/pytorch-lightning","slug":"the-optimizer-does-not-seem-to-reference-any-fsdp-704770","errorCode":null,"errorMessage":"The optimizer does not seem to reference any FSDP parameters. HINT: Make sure to create the optimizer after setting up the model by referencing `self.trainer.model.parameters()` in the `configure_optimizers()` hook.","messagePattern":"The optimizer does not seem to reference any FSDP parameters\\. HINT: Make sure to create the optimizer after setting up the model by referencing `self\\.trainer\\.model\\.parameters\\(\\)` in the `configure_optimizers\\(\\)` hook\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"src/lightning/pytorch/strategies/fsdp.py","lineNumber":388,"sourceCode":"        # and subsequent checkpoint saving can fail\n        self._reset_optimizers_and_schedulers()\n\n        if self.kwargs.get(\"use_orig_params\"):\n            return super().setup_optimizers(trainer)\n\n        invalid_params_error = False\n        try:\n            # If `use_orig_params=False` the user needs to do access `self.trainer.model.parameters()` in\n            # `configure_optimizers()`\n            super().setup_optimizers(trainer)\n        except ValueError as ex:\n            if \"optimizer got an empty parameter list\" not in str(ex):\n                raise\n            invalid_params_error = True\n\n        if invalid_params_error or any(not _optimizer_has_flat_params(optimizer) for optimizer in self.optimizers):\n            # We avoid this limitation by setting `use_orig_params=True`\n            raise ValueError(\n                \"The optimizer does not seem to reference any FSDP parameters. HINT: Make sure to create the\"\n                \" optimizer after setting up the model by referencing `self.trainer.model.parameters()` in the\"\n                \" `configure_optimizers()` hook.\"\n            )\n        return None\n\n    @override\n    def model_to_device(self) -> None:\n        # FSDP takes care of moving the model to device\n        pass\n\n    @contextmanager\n    @override\n    def tensor_init_context(self, empty_init: Optional[bool] = None) -> Generator[None, None, None]:\n        # Materialization happens in `setup`. When modules get wrapped by FSDP, the sequence of operations is:\n        # 1) materialize module 2) call `reset_parameters()` 3) shard the module.\n        # These operations are applied to each submodule 'bottom up' in the module hierarchy.\n        empty_init_context = torch.device(\"meta\") if empty_init else nullcontext()","sourceCodeStart":370,"sourceCodeEnd":406,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/strategies/fsdp.py#L370-L406","documentation":"FSDP flattens parameters into its own FlatParameter objects during setup; optimizers created over pre-wrap parameters end up with empty/foreign parameter lists. setup_optimizers detects this (torch raises 'optimizer got an empty parameter list' or `_optimizer_has_flat_params` is False) and tells you to create the optimizer after model setup using `self.trainer.model.parameters()`.","triggerScenarios":"`configure_optimizers()` referencing `self.parameters()`/`self.layer.parameters()` directly in a module whose parameters get flattened by FSDP when `use_orig_params=False` (the default in this integration path).","commonSituations":"Standard LightningModules (optimizer over self.parameters()) failing only when switched to FSDP; enabling activation checkpointing or auto-wrap policies that trigger the flattened-param path.","solutions":["In `configure_optimizers`, build the optimizer from the wrapped model: `torch.optim.AdamW(self.trainer.model.parameters())` (fallback to `self.parameters()` if trainer/model unavailable, e.g. `self.trainer is not None` branch)","Or construct `FSDPStrategy(..., use_orig_params=True)` so original parameters stay visible to the optimizer","Ensure configure_optimizers runs after setup (Lightning guarantees this when the hook is deferred correctly)"],"exampleFix":"# before\ndef configure_optimizers(self):\n    return torch.optim.AdamW(self.parameters(), lr=1e-4)\n\n# after\ndef configure_optimizers(self):\n    params = self.trainer.model.parameters() if self.trainer else self.parameters()\n    return torch.optim.AdamW(params, lr=1e-4)","handlingStrategy":"validation","validationCode":"# in configure_optimizers\nparams = self.trainer.model.parameters() if (self.trainer and self.trainer.model is not None) else self.parameters()\noptimizer = torch.optim.AdamW(params, lr=1e-4)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Reference self.trainer.model.parameters() in configure_optimizers under FSDP","Or create FSDPStrategy with use_orig_params=True","Test one fit step on a tiny batch after switching strategies"],"tags":["fsdp","optimizer","empty-parameter-list","use-orig-params"],"backgroundTag":"fsdp-optimizer-parameter-mismatch","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}