Lightning-AI/pytorch-lightning · error · ValueError

The strategy `{FSDPStrategy.strategy_name}` requires a GPU a

Error message

The strategy `{FSDPStrategy.strategy_name}` requires a GPU accelerator, but received `accelerator={self._accelerator_flag!r}`. Please set `accelerator='cuda'`, `accelerator='gpu'`, or pass a `CUDAAccelerator()` instance to use FSDP.

What it means

FSDPShardedStrategy (FSDP) in this Lightning version is CUDA-only. If you request FSDP (by name or FSDPStrategy instance) with an accelerator other than cuda/gpu/CUDAAccelerator, a ValueError is raised before strategy setup.

Source

Thrown at src/lightning/pytorch/trainer/connectors/accelerator_connector.py:434

            else:
                device = "cpu"
            # TODO: lazy initialized device, then here could be self._strategy_flag = "single_device"
            return SingleDeviceStrategy(device=device)  # type: ignore
        if len(self._parallel_devices) > 1 and _IS_INTERACTIVE:
            return "ddp_fork"
        return "ddp"

    def _check_strategy_and_fallback(self) -> None:
        """Checks edge cases when the strategy selection was a string input, and we need to fall back to a different
        choice depending on other parameters or the environment."""
        # current fallback and check logic only apply to user pass in str config and object config
        # TODO this logic should apply to both str and object config
        strategy_flag = "" if isinstance(self._strategy_flag, Strategy) else self._strategy_flag

        if (
            strategy_flag in FSDPStrategy.get_registered_strategies() or type(self._strategy_flag) is FSDPStrategy
        ) and not (self._accelerator_flag in ("cuda", "gpu") or isinstance(self._accelerator_flag, CUDAAccelerator)):
            raise ValueError(
                f"The strategy `{FSDPStrategy.strategy_name}` requires a GPU accelerator, but received "
                f"`accelerator={self._accelerator_flag!r}`. Please set `accelerator='cuda'`, `accelerator='gpu'`,"
                " or pass a `CUDAAccelerator()` instance to use FSDP."
            )
        if strategy_flag in _DDP_FORK_ALIASES and "fork" not in torch.multiprocessing.get_all_start_methods():
            raise ValueError(
                f"You selected `Trainer(strategy='{strategy_flag}')` but process forking is not supported on this"
                f" platform. We recommend `Trainer(strategy='ddp_spawn')` instead."
            )
        if strategy_flag:
            self._strategy_flag = strategy_flag

    def _init_strategy(self) -> None:
        """Instantiate the Strategy given depending on the setting of ``_strategy_flag``."""
        # The validation of `_strategy_flag` already happened earlier on in the connector
        assert isinstance(self._strategy_flag, (str, Strategy))
        if isinstance(self._strategy_flag, str):
            self.strategy = StrategyRegistry.get(self._strategy_flag)

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Ensure a GPU is used: accelerator='cuda' (and verify torch.cuda.is_available())
2. For CPU debugging, switch strategy to 'ddp' or None and revisit FSDP only on GPU
3. For non-CUDA sharding, consider DeepSpeed or other strategies supporting your backend

Example fix

# before
trainer = Trainer(strategy="fsdp", accelerator="cpu")
# after
trainer = Trainer(strategy="fsdp", accelerator="cuda")  # requires available GPUs

import torch
if strategy in ("fsdp", FSDPStrategy) and not torch.cuda.is_available():
    strategy = "ddp"  # or raise early with a clear message
trainer = Trainer(strategy=strategy, accelerator=accelerator)

def fsdp_usable(strategy) -> bool:
    import torch
    from lightning.pytorch.strategies import FSDPStrategy
    name = strategy if isinstance(strategy, str) else type(strategy).__name__
    return not ("fsdp" in name.lower()) or torch.cuda.is_available()

try:
    trainer = Trainer(strategy="fsdp", accelerator=accelerator)
except ValueError as e:
    if "requires a GPU accelerator" in str(e):
        trainer = Trainer(strategy="ddp", accelerator=accelerator)
    else:
        raise

When it happens

Trigger: Trainer(strategy='fsdp', accelerator='cpu') or strategy=FSDPStrategy() with accelerator='tpu'/'mps'/'hpu' or auto-resolution to a non-GPU accelerator.

Common situations: Running FSDP configs on CPU for debugging; auto accelerator falling back to CPU on machines without GPUs; adapting FSDP tutorials to TPU/MPS.

Related errors

• The optimizer has references to the model's meta-device para
• `precision={precision!r})` is not supported in FSDP. `precis
• `precision={precision!r}` does not use a scaler, found {scal
• Gradient clipping is not implemented for optimizers handling
• Found multiple FSDP models in the given state. Saving checkp

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/6817d745331ceee2. Report an issue: GitHub.