Lightning-AI/pytorch-lightning · error · ValueError

You set `{type(self).__name__}(use_orig_params=False)` but t

Error message

You set `{type(self).__name__}(use_orig_params=False)` but this is not supported when setting the model and optimizer up jointly. Either set it to `True` or set the objects up in this order: Create the model, call `setup_module`, create the optimizer, call `setup_optimizer`.

What it means

setup_module_and_optimizers wraps model+optimizer jointly and requires use_orig_params=True so the optimizer keeps referencing the original parameters. If the user explicitly set use_orig_params=False in FSDP kwargs, this joint setup is impossible and ValueError is raised.

Source

Thrown at src/lightning/fabric/strategies/fsdp.py:275

    def setup_environment(self) -> None:
        super().setup_environment()
        self._setup_distributed()

        # if 'device_mesh' in the `_fsdp_kwargs` is provided as a tuple, update it into the `DeviceMesh` object here
        if isinstance(self._fsdp_kwargs.get("device_mesh"), tuple):
            from torch.distributed.device_mesh import init_device_mesh

            self._fsdp_kwargs["device_mesh"] = init_device_mesh("cuda", self._fsdp_kwargs["device_mesh"])

    @override
    def setup_module_and_optimizers(
        self, module: Module, optimizers: list[Optimizer], scheduler: Optional["_LRScheduler"] = None
    ) -> tuple[Module, list[Optimizer], Optional["_LRScheduler"]]:
        """Wraps the model into a :class:`~torch.distributed.fsdp.fully_sharded_data_parallel.FullyShardedDataParallel`
        module and sets `use_orig_params=True` to keep the reference to the original parameters in the optimizer."""
        use_orig_params = self._fsdp_kwargs.get("use_orig_params")
        if use_orig_params is False:
            raise ValueError(
                f"You set `{type(self).__name__}(use_orig_params=False)` but this is not supported when"
                " setting the model and optimizer up jointly. Either set it to `True` or set the objects"
                " up in this order: Create the model, call `setup_module`, create the optimizer,"
                " call `setup_optimizer`."
            )
        module = self.setup_module(module)
        return module, optimizers, scheduler

    @override
    def setup_module(self, module: Module) -> Module:
        """Wraps the model into a :class:`~torch.distributed.fsdp.fully_sharded_data_parallel.FullyShardedDataParallel`
        module."""
        from torch.distributed.fsdp import FullyShardedDataParallel

        if any(isinstance(mod, FullyShardedDataParallel) for mod in module.modules()):
            # The user has wrapped their submodules manually, don't apply the auto wrap policy.
            if _has_meta_device_parameters_or_buffers(module):
                rank_zero_warn(

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Set use_orig_params=True (or omit it)
2. Or change ordering: create model, call fabric.setup_module(model), create the optimizer, then fabric.setup_optimizer(optimizer)

Example fix

# before
strategy = FSDPStrategy(use_orig_params=False)
model, opt = fabric.setup(model, optimizer)

# after
strategy = FSDPStrategy(use_orig_params=True)
model, opt = fabric.setup(model, optimizer)

strategy = FSDPStrategy(use_orig_params=True)
assert strategy._fsdp_kwargs.get("use_orig_params") is not False
model, opt = fabric.setup(model, optimizer)

When it happens

Trigger: FSDPStrategy(use_orig_params=False) followed by fabric.setup(model, optimizer) (which routes to setup_module_and_optimizers).

Common situations: Copying FSDP kwargs from a script that set up model and optimizer separately; disabling use_orig_params for torch.compile or state-dict compatibility reasons and then switching to joint setup.

Related errors

• The optimizer does not seem to reference any FSDP parameters
• An optimizer should be passed only once to the `setup` metho
• The optimizer does not seem to reference any FSDP parameters
• Loading a single optimizer object from a checkpoint is not s
• Could not find a FSDP model in the provided checkpoint state

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/9552f558bb95b211. Report an issue: GitHub.