{"record":{"id":"b150c967c548fd11","repo":"Lightning-AI/pytorch-lightning","slug":"the-deepspeed-strategy-is-only-supported-on-cuda-g","errorCode":null,"errorMessage":"The DeepSpeed strategy is only supported on CUDA GPUs but `{self.accelerator.__class__.__name__}` is used.","messagePattern":"The DeepSpeed strategy is only supported on CUDA GPUs but `(.+?)` is used\\.","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"src/lightning/fabric/strategies/deepspeed.py","lineNumber":643,"sourceCode":"        \"\"\"\n        import deepspeed\n\n        model_parameters = filter(lambda p: p.requires_grad, model.parameters())\n        deepspeed_engine, deepspeed_optimizer, _, deepspeed_scheduler = deepspeed.initialize(\n            args=argparse.Namespace(device_rank=self.root_device.index),\n            config=self.config,\n            model=model,\n            model_parameters=model_parameters,\n            optimizer=optimizer,\n            lr_scheduler=scheduler,\n            dist_init_required=False,\n        )\n        return deepspeed_engine, deepspeed_optimizer, deepspeed_scheduler\n\n    @override\n    def setup_environment(self) -> None:\n        if not isinstance(self.accelerator, CUDAAccelerator):\n            raise RuntimeError(\n                f\"The DeepSpeed strategy is only supported on CUDA GPUs but `{self.accelerator.__class__.__name__}`\"\n                \" is used.\"\n            )\n        super().setup_environment()\n\n    @override\n    def _setup_distributed(self) -> None:\n        assert self.parallel_devices is not None\n        _validate_device_index_selection(self.parallel_devices)\n        reset_seed()\n        self._set_world_ranks()\n        self._init_deepspeed_distributed()\n        if not self._config_initialized:\n            self._format_config()\n            self._config_initialized = True\n\n    def _init_deepspeed_distributed(self) -> None:\n        import deepspeed","sourceCodeStart":625,"sourceCodeEnd":661,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/strategies/deepspeed.py#L625-L661","documentation":"DeepSpeedStrategy requires a CUDA GPU accelerator. During setup_environment it checks that self.accelerator is a CUDAAccelerator and raises RuntimeError otherwise, because the DeepSpeed engine (ZeRO, its optimizers, etc.) is GPU-only in this integration.","triggerScenarios":"Constructing Fabric(accelerator=\"cpu\"|\"mps\"|\"tpu\", strategy=\"deepspeed\") or passing a non-CUDA Accelerator together with DeepSpeedStrategy, then running setup.","commonSituations":"Forgetting to set the accelerator and defaulting to CPU on a machine without GPUs; trying to run DeepSpeed unit tests on CPU/Mac; explicitly requesting accelerator=\"cpu\" while keeping strategy=\"deepspeed\" from an experiment.","solutions":["Switch to a CUDA setup: Fabric(accelerator=\"cuda\", strategy=\"deepspeed\") on a GPU machine","If no GPU is available, use a different strategy, e.g. Fabric(strategy=\"ddp\") or the default parallel strategy","Ensure CUDA is actually visible (torch.cuda.is_available()) so the CUDAAccelerator gets selected"],"exampleFix":"# before\nfabric = Fabric(accelerator=\"cpu\", strategy=\"deepspeed\")\n\n# after\nfabric = Fabric(accelerator=\"cuda\", strategy=\"deepspeed\")","handlingStrategy":"validation","validationCode":"import torch\nif not torch.cuda.is_available():\n    strategy = \"ddp\"  # or \"auto\"\nelse:\n    strategy = DeepSpeedStrategy(config=cfg)\nfabric = Fabric(accelerator=\"cuda\" if torch.cuda.is_available() else \"cpu\", strategy=strategy)","typeGuard":"from lightning.fabric.accelerators import CUDAAccelerator\ndef deepspeed_ok(fabric) -> bool:\n    return isinstance(fabric.strategy.accelerator, CUDAAccelerator)","tryCatchPattern":null,"preventionTips":["Gate DeepSpeed on torch.cuda.is_available() in launch scripts","Add a CI CPU job with a non-DeepSpeed strategy"],"tags":["deepspeed","accelerator","cuda","gpu-required"],"backgroundTag":"device-not-supported","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}