{"record":{"id":"55d5f7148c84e4f4","repo":"Lightning-AI/pytorch-lightning","slug":"loading-a-single-optimizer-object-from-a-checkpoin","errorCode":null,"errorMessage":"Loading a single optimizer object from a checkpoint is not supported yet with the FSDP strategy.","messagePattern":"Loading a single optimizer object from a checkpoint is not supported yet with the FSDP strategy\\.","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/strategies/fsdp.py","lineNumber":547,"sourceCode":"    ) -> dict[str, Any]:\n        \"\"\"Load the contents from a checkpoint and restore the state of the given objects.\"\"\"\n        if not state:\n            raise ValueError(\n                f\"Got FSDPStrategy.load_checkpoint(..., state={state!r}) but a state with at least \"\n                f\" a model instance to reload is required. Pass it in like so:\"\n                \" FSDPStrategy.load_checkpoint(..., state={'model': model, ...})\"\n            )\n        # broadcast the path from rank 0 to ensure all the states are loaded from a common path\n        path = _resolve_path(self.broadcast(path))\n\n        if isinstance(state, Module):\n            from lightning.fabric.strategies.model_parallel import _load_raw_module_state_from_path\n\n            _load_raw_module_state_from_path(path, module=state, world_size=self.world_size, strict=strict)\n            return {}\n\n        if isinstance(state, Optimizer):\n            raise NotImplementedError(\n                \"Loading a single optimizer object from a checkpoint is not supported yet with the FSDP strategy.\"\n            )\n\n        from torch.distributed.checkpoint.optimizer import load_sharded_optimizer_state_dict\n        from torch.distributed.fsdp import FullyShardedDataParallel as FSDP\n\n        modules = {key: module for key, module in state.items() if _has_fsdp_modules(module)}\n        if len(modules) == 0:\n            raise ValueError(\n                \"Could not find a FSDP model in the provided checkpoint state. Please provide the model as\"\n                \" part of the state like so: `load_checkpoint(..., state={'model': model, ...})`. Make sure\"\n                \" you set up the model (and optimizers if any) through the strategy before loading the checkpoint.\"\n            )\n        optimizers = {key: optim for key, optim in state.items() if isinstance(optim, Optimizer)}\n        if len(modules) > 1:\n            raise ValueError(\n                \"Found multiple FSDP models in the given state. Loading checkpoints with FSDP is\"\n                \" currently limited to a single model per checkpoint. To load multiple models, call the\"","sourceCodeStart":529,"sourceCodeEnd":565,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/strategies/fsdp.py#L529-L565","documentation":"The FSDP strategy can load a single bare module (raw module state) and can load optimizers when they are part of a state dict, but loading one standalone Optimizer object is not implemented because sharded optimizer state needs its paired module context.","triggerScenarios":"Calling strategy.load_checkpoint(path, optimizer) where optimizer is a torch Optimizer instance rather than a Module or a dict.","commonSituations":"Resuming only optimizer state (e.g. for frozen-feature fine-tuning); porting resume logic from DDPStrategy where a lone optimizer was acceptable.","solutions":["Pass model and optimizer together: load_checkpoint(path, {'model': model, 'optimizer': optimizer})","Load the module first with load_checkpoint(path, model), then restore optimizer state via torch.distributed.checkpoint.optimizer.load_sharded_optimizer_state_dict manually","Watch the repo for the upstream feature that implements single-optimizer loading"],"exampleFix":"# before\nstrategy.load_checkpoint(path, optimizer)\n# after\nstrategy.load_checkpoint(path, state={'model': model, 'optimizer': optimizer})","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"from torch.optim import Optimizer\n\ndef is_single_optimizer(state) -> bool:\n    return isinstance(state, Optimizer)","tryCatchPattern":"try:\n    strategy.load_checkpoint(path, state)\nexcept NotImplementedError:\n    # fall back to combined model+optimizer state\n    strategy.load_checkpoint(path, state={'model': model, 'optimizer': state})","preventionTips":["Always bundle the optimizer with its model in the state dict","Check the strategy's documented limitations before resuming optimizer-only state"],"tags":["fsdp","optimizer","checkpoint","not-implemented","pytorch-lightning"],"backgroundTag":"unsupported-operation","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}