Lightning-AI/pytorch-lightning · critical · RuntimeError

The DeepSpeed strategy is only supported on CUDA GPUs but `{

Error message

The DeepSpeed strategy is only supported on CUDA GPUs but `{self.accelerator.__class__.__name__}` is used.

What it means

DeepSpeedStrategy requires a CUDA GPU accelerator. During setup_environment it checks that self.accelerator is a CUDAAccelerator and raises RuntimeError otherwise, because the DeepSpeed engine (ZeRO, its optimizers, etc.) is GPU-only in this integration.

Source

Thrown at src/lightning/fabric/strategies/deepspeed.py:643

        """
        import deepspeed

        model_parameters = filter(lambda p: p.requires_grad, model.parameters())
        deepspeed_engine, deepspeed_optimizer, _, deepspeed_scheduler = deepspeed.initialize(
            args=argparse.Namespace(device_rank=self.root_device.index),
            config=self.config,
            model=model,
            model_parameters=model_parameters,
            optimizer=optimizer,
            lr_scheduler=scheduler,
            dist_init_required=False,
        )
        return deepspeed_engine, deepspeed_optimizer, deepspeed_scheduler

    @override
    def setup_environment(self) -> None:
        if not isinstance(self.accelerator, CUDAAccelerator):
            raise RuntimeError(
                f"The DeepSpeed strategy is only supported on CUDA GPUs but `{self.accelerator.__class__.__name__}`"
                " is used."
            )
        super().setup_environment()

    @override
    def _setup_distributed(self) -> None:
        assert self.parallel_devices is not None
        _validate_device_index_selection(self.parallel_devices)
        reset_seed()
        self._set_world_ranks()
        self._init_deepspeed_distributed()
        if not self._config_initialized:
            self._format_config()
            self._config_initialized = True

    def _init_deepspeed_distributed(self) -> None:
        import deepspeed

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Switch to a CUDA setup: Fabric(accelerator="cuda", strategy="deepspeed") on a GPU machine
2. If no GPU is available, use a different strategy, e.g. Fabric(strategy="ddp") or the default parallel strategy
3. Ensure CUDA is actually visible (torch.cuda.is_available()) so the CUDAAccelerator gets selected

Example fix

# before
fabric = Fabric(accelerator="cpu", strategy="deepspeed")

# after
fabric = Fabric(accelerator="cuda", strategy="deepspeed")

import torch
if not torch.cuda.is_available():
    strategy = "ddp"  # or "auto"
else:
    strategy = DeepSpeedStrategy(config=cfg)
fabric = Fabric(accelerator="cuda" if torch.cuda.is_available() else "cpu", strategy=strategy)

from lightning.fabric.accelerators import CUDAAccelerator
def deepspeed_ok(fabric) -> bool:
    return isinstance(fabric.strategy.accelerator, CUDAAccelerator)

When it happens

Trigger: Constructing Fabric(accelerator="cpu"|"mps"|"tpu", strategy="deepspeed") or passing a non-CUDA Accelerator together with DeepSpeedStrategy, then running setup.

Common situations: Forgetting to set the accelerator and defaulting to CPU on a machine without GPUs; trying to run DeepSpeed unit tests on CPU/Mac; explicitly requesting accelerator="cpu" while keeping strategy="deepspeed" from an experiment.

Related errors

• The DeepSpeed strategy is only supported on CUDA GPUs but `{
• The selected device indices {selected_device_indices!r} don'
• accelerator set through both strategy class and accelerator
• CPU parallel_devices set through {self._strategy_flag.__clas
• GPU parallel_devices set through {self._strategy_flag.__clas

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/b150c967c548fd11. Report an issue: GitHub.