Lightning-AI/pytorch-lightning · critical · RuntimeError

To use Fabric with more than one device, you must call `.lau

Error message

To use Fabric with more than one device, you must call `.launch()` or use the CLI: `fabric run --help`.

What it means

Collective/parallel operations (setup, barrier, broadcast, all_gather, all_reduce, init_module) require that processes were actually launched. _validate_launched() checks self._launched and allows only SingleDeviceStrategy and DataParallelStrategy through; any distributed strategy used without .launch() (or the CLI) raises RuntimeError.

Source

Thrown at src/lightning/fabric/fabric.py:1195

    def _get_distributed_sampler(dataloader: DataLoader, **kwargs: Any) -> DistributedSampler:
        kwargs.setdefault("shuffle", isinstance(dataloader.sampler, RandomSampler))
        kwargs.setdefault("seed", int(os.getenv("PL_GLOBAL_SEED", 0)))
        if isinstance(dataloader.sampler, (RandomSampler, SequentialSampler)):
            return DistributedSampler(dataloader.dataset, **kwargs)
        return DistributedSamplerWrapper(dataloader.sampler, **kwargs)

    def _prepare_run_method(self) -> None:
        if is_overridden("run", self, Fabric) and _is_using_cli():
            raise TypeError(
                "Overriding `Fabric.run()` and launching from the CLI is not allowed. Run the script normally,"
                " or change your code to directly call `fabric = Fabric(...); fabric.setup(...)` etc."
            )
        # wrap the run method, so we can inject setup logic or spawn processes for the user
        setattr(self, "run", partial(self._wrap_and_launch, self.run))

    def _validate_launched(self) -> None:
        if not self._launched and not isinstance(self._strategy, (SingleDeviceStrategy, DataParallelStrategy)):
            raise RuntimeError(
                "To use Fabric with more than one device, you must call `.launch()` or use the CLI:"
                " `fabric run --help`."
            )

    def _validate_setup(self, module: nn.Module, optimizers: Sequence[Optimizer]) -> None:
        self._validate_launched()
        if isinstance(module, _FabricModule):
            raise ValueError("A model should be passed only once to the `setup` method.")

        if any(isinstance(opt, _FabricOptimizer) for opt in optimizers):
            raise ValueError("An optimizer should be passed only once to the `setup` method.")

        if isinstance(self._strategy, FSDPStrategy) and any(
            _has_meta_device_parameters_or_buffers(optimizer) for optimizer in optimizers
        ):
            raise RuntimeError(
                "The optimizer has references to the model's meta-device parameters. Materializing them is"
                " is currently not supported unless you to set up the model and optimizer(s) separately."

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Wrap training in a function and call fabric.launch(train)
2. Or launch via CLI: fabric run script.py --devices=2
3. Or keep devices=1 / single-device strategy if you did not intend to distribute

Example fix

# before
fabric = Fabric(devices=2)
model = fabric.setup(model)
# after
fabric = Fabric(devices=2)
def train(fabric):
    model = fabric.setup(model)
fabric.launch(train)

from lightning.fabric.strategies import SingleDeviceStrategy
from lightning.fabric.plugins import DataParallelStrategy
if fabric._launched or isinstance(fabric.strategy, (SingleDeviceStrategy, DataParallelStrategy)):
    model = fabric.setup(model)
else:
    fabric.launch(train)

When it happens

Trigger: Fabric(accelerator='gpu', devices=2, strategy='ddp') followed directly by fabric.setup(model) or fabric.all_reduce(tensor) without calling fabric.launch() and without using the `fabric run` CLI.

Common situations: Scaling a single-GPU Fabric script to multiple devices by just changing devices=2; forgetting that multi-device Fabric requires a launcher entry point.

Related errors

• `num_nodes` must be a positive integer, but got {num_nodes}.
• You need to set up the model first before you can call `fabr
• This script was launched through the CLI, and processes have
• `Fabric.launch(...)` needs to be a callable, but got {functi
• `Fabric.launch(function={function})` needs to take at least

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/4772b751e8166470. Report an issue: GitHub.