Lightning-AI/pytorch-lightning · error · ValueError

CPU parallel_devices set through {self._strategy_flag.__clas

Error message

CPU parallel_devices set through {self._strategy_flag.__class__.__name__} class, but accelerator set to {self._accelerator_flag}, please choose one device type

What it means

The passed Strategy instance (e.g. DDPStrategy) has `parallel_devices` whose first device is a CPU torch.device, but the `accelerator=` flag was explicitly set to something other than "auto"/"cpu" (typically "gpu"/"cuda"). Fabric detects a device-type contradiction and refuses to continue.

Source

Thrown at src/lightning/fabric/connector.py:284

                self._accelerator_flag = self._strategy_flag._accelerator
            if self._strategy_flag._precision:
                # [RFC] handle precision plugin set up conflict?
                if self._precision_instance:
                    raise ValueError("precision set through both strategy class and plugins, choose one")
                self._precision_instance = self._strategy_flag._precision
            if self._strategy_flag._checkpoint_io:
                if self.checkpoint_io:
                    raise ValueError("checkpoint_io set through both strategy class and plugins, choose one")
                self.checkpoint_io = self._strategy_flag._checkpoint_io
            if getattr(self._strategy_flag, "cluster_environment", None):
                if self._cluster_environment_flag:
                    raise ValueError("cluster_environment set through both strategy class and plugins, choose one")
                self._cluster_environment_flag = getattr(self._strategy_flag, "cluster_environment")

            if hasattr(self._strategy_flag, "parallel_devices") and self._strategy_flag.parallel_devices:
                if self._strategy_flag.parallel_devices[0].type == "cpu":
                    if self._accelerator_flag and self._accelerator_flag not in ("auto", "cpu"):
                        raise ValueError(
                            f"CPU parallel_devices set through {self._strategy_flag.__class__.__name__} class,"
                            f" but accelerator set to {self._accelerator_flag}, please choose one device type"
                        )
                    self._accelerator_flag = "cpu"
                if self._strategy_flag.parallel_devices[0].type == "cuda":
                    if self._accelerator_flag and self._accelerator_flag not in ("auto", "cuda", "gpu"):
                        raise ValueError(
                            f"GPU parallel_devices set through {self._strategy_flag.__class__.__name__} class,"
                            f" but accelerator set to {self._accelerator_flag}, please choose one device type"
                        )
                    self._accelerator_flag = "cuda"
                self._parallel_devices = self._strategy_flag.parallel_devices

    def _check_device_config_and_set_final_flags(self, devices: Union[list[int], str, int], num_nodes: int) -> None:
        if not isinstance(num_nodes, int) or num_nodes < 1:
            raise ValueError(f"`num_nodes` must be a positive integer, but got {num_nodes}.")

        self._num_nodes_flag = num_nodes

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Remove the explicit accelerator flag and let the strategy's parallel_devices decide (it will be set to "cpu")
2. Or fix parallel_devices to match: parallel_devices=[torch.device("cuda", i) for i in range(...)] when accelerator is gpu
3. Check CUDA_VISIBLE_DEVICES / GPU availability if you expected GPU devices in the strategy

Example fix

# before
strategy = DDPStrategy(parallel_devices=[torch.device("cpu")] * 4)
fabric = Fabric(strategy=strategy, accelerator="gpu")

# after
strategy = DDPStrategy(parallel_devices=[torch.device("cuda", i) for i in range(4)])
fabric = Fabric(strategy=strategy, accelerator="gpu")

devs = getattr(strategy, "parallel_devices", None)
if devs and devs[0].type == "cpu" and accelerator not in (None, "auto", "cpu"):
    raise SystemExit(f"CPU parallel_devices conflict with accelerator={accelerator}")

When it happens

Trigger: DDPStrategy(parallel_devices=[torch.device("cpu")]*4) with Fabric(strategy=strategy, accelerator="gpu") — or reusing a CPU-configured strategy object with accelerator="cuda".

Common situations: Reusing a strategy built on a CPU-only machine or in a test on a GPU box; setting accelerator="gpu" while parallel_devices defaults to CPU devices because GPUs weren't visible (CUDA_VISIBLE_DEVICES empty); copy-pasted strategy configs across environments.

Related errors

• GPU parallel_devices set through {self._strategy_flag.__clas
• accelerator set through both strategy class and accelerator
• precision set through both strategy class and plugins, choos
• checkpoint_io set through both strategy class and plugins, c
• cluster_environment set through both strategy class and plug

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/ede0cba98d36ff17. Report an issue: GitHub.