{"record":{"id":"564d3f2e6011a2ec","repo":"Lightning-AI/pytorch-lightning","slug":"the-optimizer-has-references-to-the-model-s-meta-d","errorCode":null,"errorMessage":"The optimizer has references to the model's meta-device parameters. Materializing them is is currently not supported unless you to set up the model and optimizer(s) separately. Create and set up the model first through `model = fabric.setup_module(model)`. Then create the optimizer and set it up: `optimizer = fabric.setup_optimizers(optimizer)`.","messagePattern":"The optimizer has references to the model's meta-device parameters\\. Materializing them is is currently not supported unless you to set up the model and optimizer\\(s\\) separately\\. Create and set up the model first through `model = fabric\\.setup_module\\(model\\)`\\. Then create the optimizer and set it up: `optimizer = fabric\\.setup_optimizers\\(optimizer\\)`\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/fabric.py","lineNumber":1211,"sourceCode":"    def _validate_launched(self) -> None:\n        if not self._launched and not isinstance(self._strategy, (SingleDeviceStrategy, DataParallelStrategy)):\n            raise RuntimeError(\n                \"To use Fabric with more than one device, you must call `.launch()` or use the CLI:\"\n                \" `fabric run --help`.\"\n            )\n\n    def _validate_setup(self, module: nn.Module, optimizers: Sequence[Optimizer]) -> None:\n        self._validate_launched()\n        if isinstance(module, _FabricModule):\n            raise ValueError(\"A model should be passed only once to the `setup` method.\")\n\n        if any(isinstance(opt, _FabricOptimizer) for opt in optimizers):\n            raise ValueError(\"An optimizer should be passed only once to the `setup` method.\")\n\n        if isinstance(self._strategy, FSDPStrategy) and any(\n            _has_meta_device_parameters_or_buffers(optimizer) for optimizer in optimizers\n        ):\n            raise RuntimeError(\n                \"The optimizer has references to the model's meta-device parameters. Materializing them is\"\n                \" is currently not supported unless you to set up the model and optimizer(s) separately.\"\n                \" Create and set up the model first through `model = fabric.setup_module(model)`. Then create the\"\n                \" optimizer and set it up: `optimizer = fabric.setup_optimizers(optimizer)`.\"\n            )\n\n    def _validate_setup_module(self, module: nn.Module) -> None:\n        self._validate_launched()\n        if isinstance(module, _FabricModule):\n            raise ValueError(\"A model should be passed only once to the `setup_module` method.\")\n\n    def _validate_setup_optimizers(self, optimizers: Sequence[Optimizer]) -> None:\n        self._validate_launched()\n        if isinstance(self._strategy, (DeepSpeedStrategy, XLAStrategy)):\n            raise RuntimeError(\n                f\"The `{type(self._strategy).__name__}` requires the model and optimizer(s) to be set up jointly\"\n                \" through `.setup(model, optimizer, ...)`.\"\n            )","sourceCodeStart":1193,"sourceCodeEnd":1229,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/fabric.py#L1193-L1229","documentation":"With FSDPStrategy, if an optimizer references model parameters that still live on the meta device (init_module/to_empty flow), Fabric cannot materialize them during a joint setup. The error directs you to set up the model first (materializing parameters) and create/set up the optimizer afterwards.","triggerScenarios":"model = fabric.init_module(...) then creating an optimizer over model.parameters() (still meta) and calling fabric.setup(model, optimizer) with strategy='fsdp'.","commonSituations":"Using init_module for memory-efficient initialization (large LLMs) but constructing the optimizer before materialization; copying vanilla-Fabric init order into an FSDP script.","solutions":["Reorder: model = fabric.setup_module(model) first, then optimizer = torch.optim.Adam(model.parameters(), lr=...); optimizer = fabric.setup_optimizers(optimizer)","Ensure parameters are moved to a real device (to_empty/device) before building the optimizer"],"exampleFix":"# before\nwith fabric.init_module():\n    model = MyModel()\noptimizer = Adam(model.parameters())\nmodel, optimizer = fabric.setup(model, optimizer)  # FSDP\n# after\nwith fabric.init_module():\n    model = MyModel()\nmodel = fabric.setup_module(model)\noptimizer = fabric.setup_optimizers(Adam(model.parameters()))","handlingStrategy":"validation","validationCode":"from lightning.fabric.strategies import FSDPStrategy\nif isinstance(fabric.strategy, FSDPStrategy) and any(\n    p.is_meta for opt in optimizers for group in opt.param_groups for p in group['params']\n):\n    raise SystemExit('set up the model first, then create the optimizer')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["With init_module + FSDP: call setup_module(model) BEFORE constructing the optimizer over model.parameters()"],"tags":["lightning","fabric","fsdp","meta-device","init-module"],"backgroundTag":"meta-device-parameters","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}