{"record":{"id":"d39c7ccb88a00378","repo":"Lightning-AI/pytorch-lightning","slug":"could-not-find-a-deepspeed-model-in-the-provided-c","errorCode":null,"errorMessage":"Could not find a DeepSpeed model in the provided checkpoint state. Please provide the model as part of the state like so: `save_checkpoint(..., state={'model': model, ...})`. Make sure you set up the model (and optimizers if any) through the strategy before saving the checkpoint.","messagePattern":"Could not find a DeepSpeed model in the provided checkpoint state\\. Please provide the model as part of the state like so: `save_checkpoint\\(\\.\\.\\., state=(.+?)\\)`\\. Make sure you set up the model \\(and optimizers if any\\) through the strategy before saving the checkpoint\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/strategies/deepspeed.py","lineNumber":443,"sourceCode":"            ValueError:\n                When no :class:`deepspeed.DeepSpeedEngine` objects were found in the state, or when multiple\n                :class:`deepspeed.DeepSpeedEngine` objects were found.\n\n        \"\"\"\n        if storage_options is not None:\n            raise TypeError(\n                \"`DeepSpeedStrategy.save_checkpoint(..., storage_options=...)` is not supported because\"\n                \" `DeepSpeedStrategy` does not use the `CheckpointIO`.\"\n            )\n        if filter is not None:\n            raise TypeError(\n                \"`DeepSpeedStrategy.save_checkpoint(..., filter=...)` is not supported because\"\n                \" `DeepSpeedStrategy` manages the state serialization internally.\"\n            )\n\n        engines = _get_deepspeed_engines_from_state(state)\n        if len(engines) == 0:\n            raise ValueError(\n                \"Could not find a DeepSpeed model in the provided checkpoint state. Please provide the model as\"\n                \" part of the state like so: `save_checkpoint(..., state={'model': model, ...})`. Make sure\"\n                \" you set up the model (and optimizers if any) through the strategy before saving the checkpoint.\"\n            )\n        if len(engines) > 1:\n            raise ValueError(\n                \"Found multiple DeepSpeed engine modules in the given state. Saving checkpoints with DeepSpeed is\"\n                \" currently limited to a single model per checkpoint. To save multiple models, call the\"\n                \" save method for each model separately with a different path.\"\n            )\n        engine = engines[0]\n\n        # broadcast the path from rank 0 to ensure all the states are saved in a common path\n        path = self.broadcast(path)\n\n        # split the checkpoint into two parts:\n        # 1) the deepspeed engine encapsulating both the model and optionally the optimizer(s)\n        # 2) the rest of the user's state, which in deepspeed is called `client state`","sourceCodeStart":425,"sourceCodeEnd":461,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/strategies/deepspeed.py#L425-L461","documentation":"DeepSpeed checkpoints are saved by the DeepSpeedEngine, so the state passed to save_checkpoint must contain at least one module that was set up through the strategy (i.e. a DeepSpeedEngine instance). If _get_deepspeed_engines_from_state finds none, this ValueError is raised.","triggerScenarios":"fabric.save_checkpoint(path, state) where state contains only raw tensors, an unwrapped nn.Module, or a module created but never passed through fabric.setup/fabric.setup_module under DeepSpeedStrategy.","commonSituations":"Saving a state dict of raw weights instead of the model object; calling save_checkpoint before setup; mixing strategy objects (model set up under a different fabric instance).","solutions":["Pass the set-up model object in state: fabric.save_checkpoint(path, {'model': model, ...}) where model came from fabric.setup/fabric.setup_module","Ensure the same Fabric/strategy instance was used to set up the model","For raw-weight-only saves, use your own torch.save instead of the strategy checkpoint API"],"exampleFix":"# before\nfabric.save_checkpoint(path, state={'weights': model.state_dict()})\n# after\nmodel = fabric.setup(model, optimizer)\nfabric.save_checkpoint(path, state={'model': model, 'optimizer': optimizer})","handlingStrategy":"validation","validationCode":"from deepspeed import DeepSpeedEngine\ndef has_engine(state) -> bool:\n    return any(isinstance(v, DeepSpeedEngine) for v in state.values())\nassert has_engine(state), \"state must include the set-up model\"","typeGuard":"from deepspeed import DeepSpeedEngine\n\ndef checkpoint_state_valid(state: dict) -> bool:\n    return bool(state) and any(isinstance(v, DeepSpeedEngine) for v in state.values())","tryCatchPattern":null,"preventionTips":["Always include the fabric.setup-returned model object in checkpoint state","Never save raw state_dicts through the DeepSpeed strategy API"],"tags":["deepspeed","checkpointing","state-validation","pytorch-lightning"],"backgroundTag":"invalid-state-argument","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}