{"record":{"id":"47abba9da4b52161","repo":"Lightning-AI/pytorch-lightning","slug":"xlafsdp-only-supports-a-single-model-instance-with","errorCode":null,"errorMessage":"XLAFSDP only supports a single model instance with 'model' as the key.","messagePattern":"XLAFSDP only supports a single model instance with 'model' as the key\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/strategies/xla_fsdp.py","lineNumber":611,"sourceCode":"\n        if self._state_dict_type == \"full\":\n            if not path.is_file():\n                raise ValueError(\n                    f\"The path {str(path)!r} does not point to a valid full checkpoint. Make sure the path points to a\"\n                    \" directory with a full XLAFSDP checkpoint.\"\n                )\n            if len(optimizers) > 0 or len(state.keys() - modules.keys() - optimizers.keys()) > 0:\n                rank_zero_warn(\n                    \"Loading a full checkpoint will only load the full model.\"\n                    \" The optimizer and any additional metadata are not included.\"\n                )\n            if len(modules) > 0:\n                raise ValueError(\n                    \"Found a XLAFSDP model in the provided checkpoint state.\"\n                    \" Please provide the model without any XLAFSDP wrapper.\"\n                )\n            if \"model\" not in state or not isinstance(model := state[\"model\"], torch.nn.Module):\n                raise NotImplementedError(\"XLAFSDP only supports a single model instance with 'model' as the key.\")\n            full_ckpt = torch.load(path, weights_only=weights_only)\n            model.load_state_dict(full_ckpt.pop(\"model\"), strict=strict)\n            return full_ckpt\n\n        raise ValueError(f\"Unknown state_dict_type: {self._state_dict_type}\")\n\n    @classmethod\n    @override\n    def register_strategies(cls, strategy_registry: _StrategyRegistry) -> None:\n        strategy_registry.register(\"xla_fsdp\", cls, description=cls.__name__)\n\n    def _parse_fsdp_kwargs(self) -> dict:\n        # this needs to be delayed because `self.precision` isn't available at init\n        kwargs = self._fsdp_kwargs.copy()\n        precision = self.precision\n        if isinstance(precision, XLAPrecision):\n            # the `compute_dtype` will be passed to the `auto_wrapper_callable` automatically, so we don't need to pass\n            # it when creating it","sourceCodeStart":593,"sourceCodeEnd":629,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/strategies/xla_fsdp.py#L593-L629","documentation":"For state_dict_type='full' loading, XLAFSDP support is limited to exactly one model stored under the key 'model' in the state dict; the loader does a targeted state['model'].load_state_dict(...). Any other key name, a missing 'model' key, or a non-nn.Module value is not implemented. Optimizers and other state entries are ignored in this mode (a warning covers that).","triggerScenarios":"fabric.load_checkpoint(path, state={'net': model}) (wrong key), state={'model': state_dict_tensor} (not a Module), or state={'model': m, 'optimizer': opt} where value under 'model' is not an nn.Module.","commonSituations":"Using generic key names from other Lightning workflows; passing model.state_dict() instead of the module; expecting multi-object restore from full checkpoints.","solutions":["Use exactly state={'model': model} with an unwrapped torch.nn.Module instance","Store/restore optimizer state separately (it is not restored by full-checkpoint loading) — e.g. save optimizer state as part of the checkpoint's extra dict and apply manually"],"exampleFix":"# before\nfabric.load_checkpoint(path, state={'net': model})\n\n# after\nfabric.load_checkpoint(path, state={'model': model})","handlingStrategy":"validation","validationCode":"assert 'model' in state and hasattr(state['model'], 'load_state_dict'), \"full load requires state={'model': nn.Module}\"","typeGuard":"import torch.nn as nn\n\ndef full_state_is_valid(state: dict) -> bool:\n    return isinstance(state, dict) and isinstance(state.get('model'), nn.Module)","tryCatchPattern":null,"preventionTips":["Use the literal key 'model' for XLAFSDP full checkpoints","Save optimizer state separately; it is not restored by full loading"],"tags":["xla","fsdp","checkpoint","load","full-checkpoint","not-implemented","lightning-fabric"],"backgroundTag":"checkpoint-state-validation-failed","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}