{"record":{"id":"18235f9cfb6272c0","repo":"Lightning-AI/pytorch-lightning","slug":"found-modules-that-are-wrapped-with-torch-distrib-18235f","errorCode":null,"errorMessage":"Found modules that are wrapped with `torch.distributed.fsdp.FullyShardedDataParallel`. The `{self.__class__.__name__}` only supports the new FSDP2 APIs in PyTorch >= 2.4.","messagePattern":"Found modules that are wrapped with `torch\\.distributed\\.fsdp\\.FullyShardedDataParallel`\\. The `(.+?)` only supports the new FSDP2 APIs in PyTorch >= 2\\.4\\.","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/strategies/model_parallel.py","lineNumber":175,"sourceCode":"        # Users can access device mesh in `LightningModule.configure_model()`\n        assert self.lightning_module is not None\n        self.lightning_module._device_mesh = self._device_mesh\n\n    @override\n    def setup(self, trainer: \"pl.Trainer\") -> None:\n        from torch.distributed.fsdp import FullyShardedDataParallel\n\n        assert self.model is not None\n        assert self.accelerator is not None\n        self.accelerator.setup(trainer)\n\n        if not is_overridden(\"configure_model\", self.lightning_module):\n            raise TypeError(\n                f\"When using the {type(self).__name__}, you are required to override the `configure_model()` hook in\"\n                f\" the LightningModule and apply parallelization there.\"\n            )\n        if any(isinstance(mod, FullyShardedDataParallel) for mod in self.model.modules()):\n            raise TypeError(\n                \"Found modules that are wrapped with `torch.distributed.fsdp.FullyShardedDataParallel`.\"\n                f\" The `{self.__class__.__name__}` only supports the new FSDP2 APIs in PyTorch >= 2.4.\"\n            )\n\n        _materialize_distributed_module(self.model, self.root_device)\n\n        self.model = self.precision_plugin.convert_module(self.model)\n        self.model_to_device()  # move all remaining layers if any left on CPU.\n\n        self.barrier()\n\n        if trainer.state.fn == TrainerFn.FITTING:\n            self.setup_optimizers(trainer)\n        self.setup_precision_plugin()\n        if trainer.state.fn == TrainerFn.FITTING:\n            _optimizers_to_device(self.optimizers, self.root_device)\n\n    @override","sourceCodeStart":157,"sourceCodeEnd":193,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/strategies/model_parallel.py#L157-L193","documentation":"ModelParallelStrategy in current Lightning supports only FSDP2 (torch.distributed.fsdp.fully_shard, PyTorch >= 2.4). If it detects modules wrapped in the legacy class torch.distributed.fsdp.FullyShardedDataParallel, it raises TypeError, since mixing legacy FSDP with FSDP2-based plans is unsupported.","triggerScenarios":"Manually wrapping modules with FullyShardedDataParallel(m) inside configure_model (or anywhere) and then using ModelParallelStrategy / FSDP2-based strategies; old code migrated from FSDPStrategy (which used legacy FSDP1).","commonSituations":"Upgrading from PyTorch < 2.4 era FSDP code or Lightning's older FSDPStrategy; copy-pasted FSDP1 tutorials.","solutions":["Replace FullyShardedDataParallel(module, ...) wrapping with FSDP2's fully_shard(module) applied per-submodule","Require torch >= 2.4 and Lightning versions where FSDP2 is the path","If you must use legacy FSDP, use an older Lightning release's FSDPStrategy"],"exampleFix":"# before\nfrom torch.distributed.fsdp import FullyShardedDataParallel as FSDP\nself.model = FSDP(self.model, auto_wrap_policy=policy)\n\n# after\nfrom torch.distributed.fsdp import fully_shard\nfor block in self.model.blocks:\n    fully_shard(block)\nfully_shard(self.model)","handlingStrategy":"type-guard","validationCode":"import torch\nfrom torch.distributed.fsdp import FullyShardedDataParallel\nassert not any(isinstance(m, FullyShardedDataParallel) for m in model.modules()), \"use fully_shard (FSDP2) instead\"","typeGuard":"def uses_legacy_fsdp(model) -> bool:\n    from torch.distributed.fsdp import FullyShardedDataParallel\n    return any(isinstance(m, FullyShardedDataParallel) for m in model.modules())","tryCatchPattern":null,"preventionTips":["Use fully_shard() (FSDP2) with torch>=2.4","Search your codebase for FullyShardedDataParallel before adopting ModelParallelStrategy"],"tags":["fsdp","fsdp2","legacy-api","pytorch-version"],"backgroundTag":"legacy-fsdp-unsupported","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}