{"record":{"id":"1c06ea228832ca5e","repo":"Lightning-AI/pytorch-lightning","slug":"the-optimizer-does-not-seem-to-reference-any-fsdp","errorCode":null,"errorMessage":"The optimizer does not seem to reference any FSDP parameters. HINT: Make sure to create the optimizer after setting up the model.","messagePattern":"The optimizer does not seem to reference any FSDP parameters\\. HINT: Make sure to create the optimizer after setting up the model\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/strategies/fsdp.py","lineNumber":335,"sourceCode":"        # activation checkpointing needs to be set up after wrapping the model\n        _setup_activation_checkpointing(module, self._activation_checkpointing_kwargs)\n\n        return module\n\n    @override\n    def setup_optimizer(self, optimizer: Optimizer) -> Optimizer:\n        \"\"\"Set up an optimizer for a model wrapped with FSDP.\n\n        This setup method doesn't modify the optimizer or wrap the optimizer. The only thing it currently does is verify\n        that the optimizer was created after the model was wrapped with :meth:`setup_module` with a reference to the\n        flattened parameters.\n\n        \"\"\"\n        if self._fsdp_kwargs.get(\"use_orig_params\"):\n            return super().setup_optimizer(optimizer)\n        if not _optimizer_has_flat_params(optimizer):\n            # We avoid this limitation by setting `use_orig_params=True`\n            raise ValueError(\n                \"The optimizer does not seem to reference any FSDP parameters. HINT: Make sure to create the optimizer\"\n                \" after setting up the model.\"\n            )\n        return optimizer\n\n    @override\n    def module_to_device(self, module: Module) -> None:\n        pass\n\n    @override\n    def module_init_context(self, empty_init: Optional[bool] = None) -> AbstractContextManager:\n        precision_init_ctx = self.precision.module_init_context()\n        module_sharded_ctx = self.module_sharded_context()\n        stack = ExitStack()\n        if empty_init:\n            # Materialization happens in `setup`. When modules get wrapped by FSDP, the sequence of operations is:\n            # 1) materialize module 2) call `reset_parameters()` 3) shard the module.\n            # These operations are applied to each submodule 'bottom up' in the module hierarchy.","sourceCodeStart":317,"sourceCodeEnd":353,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/strategies/fsdp.py#L317-L353","documentation":"With use_orig_params unset/False, FSDP setup_optimizer requires the optimizer to reference the flat (flattened) FSDP parameter groups created by setup_module. _optimizer_has_flat_params detects no FlatParameter in the optimizer, meaning the optimizer was created before the model was wrapped, and raises ValueError.","triggerScenarios":"Creating torch.optim.Adam(model.parameters(), ...) on the raw model and calling fabric.setup_optimizer(optimizer) after FSDP wrapped the model with use_orig_params not True — the optimizer's params are stale references.","commonSituations":"Standard PyTorch ordering (optimizer before setup) copied into Fabric FSDP code; refactoring from DDP where param identity is preserved; partial refactor where setup_module was called but the optimizer object predates it.","solutions":["Recreate the optimizer after setup_module so it sees the FSDP parameters, or just use fabric.setup(model, optimizer)","Set FSDPStrategy(use_orig_params=True) so original parameter references stay valid","Pass module.parameters() to a new optimizer created after the module is set up"],"exampleFix":"# before\noptimizer = torch.optim.Adam(model.parameters(), lr=1e-3)\nmodel = fabric.setup_module(model)\noptimizer = fabric.setup_optimizer(optimizer)  # ValueError\n\n# after\nmodel, optimizer = fabric.setup(model, torch.optim.Adam(model.parameters(), lr=1e-3))","handlingStrategy":"fallback","validationCode":"# create optimizer only after module setup\nmodel = fabric.setup_module(model)\noptimizer = torch.optim.Adam(model.parameters(), lr=1e-3)  # now references FSDP params\noptimizer = fabric.setup_optimizer(optimizer)","typeGuard":null,"tryCatchPattern":"try:\n    optimizer = fabric.setup_optimizer(optimizer)\nexcept ValueError as e:\n    if \"does not seem to reference any FSDP parameters\" in str(e):\n        optimizer = type(optimizer)(module.parameters(), lr=optimizer.defaults[\"lr\"])\n        optimizer = fabric.setup_optimizer(optimizer)","preventionTips":["Use fabric.setup(model, optimizer) as a single call","Never reuse an optimizer built from pre-wrap parameters"],"tags":["fsdp","optimizer","flat-params","setup-order"],"backgroundTag":"optimizer-setup-order","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}