{"record":{"id":"dc85737e1c6fc8ca","repo":"Lightning-AI/pytorch-lightning","slug":"the-optimizer-has-references-to-the-model-s-meta-d-dc8573","errorCode":null,"errorMessage":"The optimizer has references to the model's meta-device parameters. Materializing them is is currently not supported. Create the optimizer after setting up the model, then call `fabric.setup_optimizers(optimizer)`.","messagePattern":"The optimizer has references to the model's meta-device parameters\\. Materializing them is is currently not supported\\. Create the optimizer after setting up the model, then call `fabric\\.setup_optimizers\\(optimizer\\)`\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/fabric.py","lineNumber":1238,"sourceCode":"        if isinstance(module, _FabricModule):\n            raise ValueError(\"A model should be passed only once to the `setup_module` method.\")\n\n    def _validate_setup_optimizers(self, optimizers: Sequence[Optimizer]) -> None:\n        self._validate_launched()\n        if isinstance(self._strategy, (DeepSpeedStrategy, XLAStrategy)):\n            raise RuntimeError(\n                f\"The `{type(self._strategy).__name__}` requires the model and optimizer(s) to be set up jointly\"\n                \" through `.setup(model, optimizer, ...)`.\"\n            )\n\n        if not optimizers:\n            raise ValueError(\"`setup_optimizers` requires at least one optimizer as input.\")\n\n        if any(isinstance(opt, _FabricOptimizer) for opt in optimizers):\n            raise ValueError(\"An optimizer should be passed only once to the `setup_optimizers` method.\")\n\n        if any(_has_meta_device_parameters_or_buffers(optimizer) for optimizer in optimizers):\n            raise RuntimeError(\n                \"The optimizer has references to the model's meta-device parameters. Materializing them is\"\n                \" is currently not supported. Create the optimizer after setting up the model, then call\"\n                \" `fabric.setup_optimizers(optimizer)`.\"\n            )\n\n    def _validate_setup_dataloaders(self, dataloaders: Sequence[DataLoader]) -> None:\n        self._validate_launched()\n        if not dataloaders:\n            raise ValueError(\"`setup_dataloaders` requires at least one dataloader as input.\")\n\n        if any(isinstance(dl, _FabricDataLoader) for dl in dataloaders):\n            raise ValueError(\"A dataloader should be passed only once to the `setup_dataloaders` method.\")\n\n        if any(not isinstance(dl, DataLoader) for dl in dataloaders):\n            raise TypeError(\"Only PyTorch DataLoader are currently supported in `setup_dataloaders`.\")\n\n    @staticmethod\n    def _configure_callbacks(callbacks: Optional[Union[list[Any], Any]]) -> list[Any]:","sourceCodeStart":1220,"sourceCodeEnd":1256,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/fabric.py#L1220-L1256","documentation":"Raised when an optimizer passed to `fabric.setup_optimizers()` references model parameters that still live on the meta device (e.g. created via `torch.device('meta')` or a factory like `with torch.device('meta'):`). Fabric cannot materialize meta parameters through this path, so the optimizer must be created after the model is set up (which materializes the parameters).","triggerScenarios":"Creating a model under `with torch.device('meta'):`, building an optimizer over `model.parameters()` while still on meta device, then calling `fabric.setup(model)` followed by `fabric.setup_optimizers(optimizer)`. The check `_has_meta_device_parameters_or_buffers` finds meta-device params in the optimizer state.","commonSituations":"Using meta-device initialization to avoid allocating memory twice for large models (FSDP/deferred init workflows) and keeping the old combined setup call order; upgrading Fabric versions where split setup became required for this pattern.","solutions":["Create the optimizer AFTER `model = fabric.setup(model)` so parameters are materialized, then call `fabric.setup_optimizers(optimizer)`","Ensure the strategy you use supports meta-device init at all; for strategies requiring joint setup, use `fabric.setup(model, optimizer)` instead of the split path"],"exampleFix":"# before\nwith torch.device('meta'):\n    model = BigModel()\noptimizer = torch.optim.Adam(model.parameters(), lr=1e-3)  # meta params\nmodel = fabric.setup(model)\nfabric.setup_optimizers(optimizer)\n\n# after\nwith torch.device('meta'):\n    model = BigModel()\nmodel = fabric.setup(model)  # materializes params\noptimizer = torch.optim.Adam(model.parameters(), lr=1e-3)\nfabric.setup_optimizers(optimizer)","handlingStrategy":"validation","validationCode":"def has_meta_params(module) -> bool:\n    return any(p.is_meta for p in module.parameters())\nassert not has_meta_params(model), 'materialize model (fabric.setup) before creating optimizer'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always create optimizers after fabric.setup(model) when using meta-device init","Document call order in meta-init training scripts"],"tags":["pytorch-lightning","fabric","meta-device","fsdp","lazy-init"],"backgroundTag":"meta-device-parameters","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}