Lightning-AI/pytorch-lightning · critical · ValueError

The optimizer does not seem to reference any FSDP parameters

Error message

The optimizer does not seem to reference any FSDP parameters. HINT: Make sure to create the optimizer after setting up the model by referencing `self.trainer.model.parameters()` in the `configure_optimizers()` hook.

What it means

FSDP flattens parameters into its own FlatParameter objects during setup; optimizers created over pre-wrap parameters end up with empty/foreign parameter lists. setup_optimizers detects this (torch raises 'optimizer got an empty parameter list' or `_optimizer_has_flat_params` is False) and tells you to create the optimizer after model setup using `self.trainer.model.parameters()`.

Source

Thrown at src/lightning/pytorch/strategies/fsdp.py:388

        # and subsequent checkpoint saving can fail
        self._reset_optimizers_and_schedulers()

        if self.kwargs.get("use_orig_params"):
            return super().setup_optimizers(trainer)

        invalid_params_error = False
        try:
            # If `use_orig_params=False` the user needs to do access `self.trainer.model.parameters()` in
            # `configure_optimizers()`
            super().setup_optimizers(trainer)
        except ValueError as ex:
            if "optimizer got an empty parameter list" not in str(ex):
                raise
            invalid_params_error = True

        if invalid_params_error or any(not _optimizer_has_flat_params(optimizer) for optimizer in self.optimizers):
            # We avoid this limitation by setting `use_orig_params=True`
            raise ValueError(
                "The optimizer does not seem to reference any FSDP parameters. HINT: Make sure to create the"
                " optimizer after setting up the model by referencing `self.trainer.model.parameters()` in the"
                " `configure_optimizers()` hook."
            )
        return None

    @override
    def model_to_device(self) -> None:
        # FSDP takes care of moving the model to device
        pass

    @contextmanager
    @override
    def tensor_init_context(self, empty_init: Optional[bool] = None) -> Generator[None, None, None]:
        # Materialization happens in `setup`. When modules get wrapped by FSDP, the sequence of operations is:
        # 1) materialize module 2) call `reset_parameters()` 3) shard the module.
        # These operations are applied to each submodule 'bottom up' in the module hierarchy.
        empty_init_context = torch.device("meta") if empty_init else nullcontext()

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. In `configure_optimizers`, build the optimizer from the wrapped model: `torch.optim.AdamW(self.trainer.model.parameters())` (fallback to `self.parameters()` if trainer/model unavailable, e.g. `self.trainer is not None` branch)
2. Or construct `FSDPStrategy(..., use_orig_params=True)` so original parameters stay visible to the optimizer
3. Ensure configure_optimizers runs after setup (Lightning guarantees this when the hook is deferred correctly)

Example fix

# before
def configure_optimizers(self):
    return torch.optim.AdamW(self.parameters(), lr=1e-4)

# after
def configure_optimizers(self):
    params = self.trainer.model.parameters() if self.trainer else self.parameters()
    return torch.optim.AdamW(params, lr=1e-4)

# in configure_optimizers
params = self.trainer.model.parameters() if (self.trainer and self.trainer.model is not None) else self.parameters()
optimizer = torch.optim.AdamW(params, lr=1e-4)

When it happens

Trigger: `configure_optimizers()` referencing `self.parameters()`/`self.layer.parameters()` directly in a module whose parameters get flattened by FSDP when `use_orig_params=False` (the default in this integration path).

Common situations: Standard LightningModules (optimizer over self.parameters()) failing only when switched to FSDP; enabling activation checkpointing or auto-wrap policies that trigger the flattened-param path.

Related errors

• You set `{type(self).__name__}(use_orig_params=False)` but t
• The optimizer does not seem to reference any FSDP parameters
• Loading a single optimizer object from a checkpoint is not s
• An optimizer should be passed only once to the `setup` metho
• The optimizer has references to the model's meta-device para

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/7047705a10f68b3a. Report an issue: GitHub.