Lightning-AI/pytorch-lightning · critical · RuntimeError

The DeepSpeed strategy is only supported on CUDA GPUs but `{

Error message

The DeepSpeed strategy is only supported on CUDA GPUs but `{self.accelerator.__class__.__name__}` is used.

What it means

DeepSpeed's engine only accelerates on NVIDIA CUDA GPUs, so DeepSpeedStrategy.setup_environment validates that the configured accelerator is CUDAAccelerator and raises RuntimeError otherwise. This surfaces early (during environment setup) instead of failing obscurely inside DeepSpeed kernels.

Source

Thrown at src/lightning/pytorch/strategies/deepspeed.py:343

        self._config_initialized = False
        deepspeed.utils.logging.logger.setLevel(logging_level)

        self.remote_device = remote_device
        self.load_full_weights = load_full_weights
        self.exclude_frozen_parameters = exclude_frozen_parameters

        # default FP16 parameters.
        self.loss_scale = loss_scale
        self.initial_scale_power = initial_scale_power
        self.loss_scale_window = loss_scale_window
        self.hysteresis = hysteresis
        self.min_loss_scale = min_loss_scale

    @override
    def setup_environment(self) -> None:
        if not isinstance(self.accelerator, CUDAAccelerator):
            raise RuntimeError(
                f"The DeepSpeed strategy is only supported on CUDA GPUs but `{self.accelerator.__class__.__name__}`"
                " is used."
            )
        super().setup_environment()

    @override
    def setup_distributed(self) -> None:
        assert self.parallel_devices is not None
        _validate_device_index_selection(self.parallel_devices)
        reset_seed()
        self.set_world_ranks()
        self._init_deepspeed_distributed()

    @override
    def setup(self, trainer: "pl.Trainer") -> None:
        self._init_config_if_needed()
        assert self.accelerator is not None
        self.accelerator.setup(trainer)

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Set `accelerator="gpu", devices=...` and run on a CUDA machine
2. If you only need CPU, use the default single-device strategy instead of DeepSpeed
3. Gate DeepSpeed usage on GPU availability before building the Trainer

Example fix

# before
trainer = Trainer(strategy=DeepSpeedStrategy(), accelerator="auto")  # resolves to CPU

# after
trainer = Trainer(strategy=DeepSpeedStrategy(), accelerator="gpu", devices=2)

import torch
use_deepspeed = torch.cuda.is_available()
strategy = DeepSpeedStrategy(...) if use_deepspeed else "auto"

When it happens

Trigger: `Trainer(strategy=DeepSpeedStrategy(...), accelerator="cpu")` or `accelerator="tpu"`/`accelerator="mps"` — any non-CUDA accelerator reaching `setup_environment()`. Also when devices are configured such that Lightning auto-selects a non-CUDA accelerator.

Common situations: Testing a DeepSpeed config on a CPU-only laptop/CI box; copy-pasted Trainer flags with `accelerator` left at default on a machine without GPUs; MPS Macs.

Related errors

• The DeepSpeed strategy is only supported on CUDA GPUs but `{
• The selected device indices {selected_device_indices!r} don'
• accelerator set through both strategy class and accelerator
• CPU parallel_devices set through {self._strategy_flag.__clas
• GPU parallel_devices set through {self._strategy_flag.__clas

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/570aa3916858876c. Report an issue: GitHub.