{"record":{"id":"238fa29f7ba61750","repo":"Lightning-AI/pytorch-lightning","slug":"bitsandbytes-is-only-supported-on-cuda-gpus-238fa2","errorCode":null,"errorMessage":"Bitsandbytes is only supported on CUDA GPUs.","messagePattern":"Bitsandbytes is only supported on CUDA GPUs\\.","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/trainer/connectors/accelerator_connector.py","lineNumber":499,"sourceCode":"                \"CPU. Using `precision='bf16-mixed'` instead.\"\n            )\n            self._precision_flag = \"bf16-mixed\"\n\n        if self._precision_flag in (\"16-mixed\", \"bf16-mixed\"):\n            rank_zero_info(\n                f\"Using {'16bit' if self._precision_flag == '16-mixed' else 'bfloat16'} Automatic Mixed Precision (AMP)\"\n            )\n            device = self._accelerator_flag if self._accelerator_flag in (\"cpu\", \"mps\") else \"cuda\"\n            return MixedPrecision(self._precision_flag, device)\n\n        raise RuntimeError(\"No precision set\")\n\n    def _validate_precision_choice(self) -> None:\n        \"\"\"Validate the combination of choices for precision, AMP type, and accelerator.\"\"\"\n        if isinstance(self._precision_plugin_flag, BitsandbytesPrecision) and not isinstance(\n            self.accelerator, CUDAAccelerator\n        ):\n            raise RuntimeError(\"Bitsandbytes is only supported on CUDA GPUs.\")\n        mp_precision_supported = (\"32-true\", \"bf16-mixed\", \"bf16-true\", \"16-true\")\n        if (\n            isinstance(self._strategy_flag, ModelParallelStrategy)\n            and self._precision_flag not in mp_precision_supported\n        ):\n            raise ValueError(\n                f\"The `ModelParallelStrategy` does not support `Fabric(..., precision={self._precision_flag!r})`.\"\n                f\" Choose a different precision among: {', '.join(mp_precision_supported)}.\"\n            )\n\n    def _lazy_init_strategy(self) -> None:\n        \"\"\"Lazily set missing attributes on the previously instantiated strategy.\"\"\"\n        self.strategy.accelerator = self.accelerator\n        if self.precision_plugin:\n            self.strategy.precision_plugin = self.precision_plugin\n        if self.checkpoint_io:\n            self.strategy.checkpoint_io = self.checkpoint_io\n        if hasattr(self.strategy, \"cluster_environment\"):","sourceCodeStart":481,"sourceCodeEnd":517,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/trainer/connectors/accelerator_connector.py#L481-L517","documentation":"BitsandbytesPrecision (quantized 8/4-bit via the bitsandbytes library) only works on CUDA GPUs. If the resolved accelerator is not a CUDAAccelerator, initialization raises RuntimeError. The check runs inside _validate_precision_choice after accelerator init.","triggerScenarios":"Trainer(plugins=[BitsandbytesPrecision(mode='q8')], accelerator='cpu') or on a machine where accelerator auto-resolves to CPU/MPS while a bitsandbytes plugin is configured.","commonSituations":"QLoRA / quantized LLM fine-tuning configs run on CPU-only machines or in CI; MPS Macs attempting bitsandbytes workflows.","solutions":["Run on a CUDA GPU: set accelerator='cuda' and ensure torch.cuda.is_available() is True","Remove the BitsandbytesPrecision plugin when CPU training is intended"],"exampleFix":"# before\ntrainer = Trainer(plugins=[BitsandbytesPrecision(mode=\"q8\")], accelerator=\"cpu\")\n# after\ntrainer = Trainer(plugins=[BitsandbytesPrecision(mode=\"q8\")], accelerator=\"cuda\")","handlingStrategy":"validation","validationCode":"import torch\nfrom lightning.pytorch.plugins import BitsandbytesPrecision\nif any(isinstance(p, BitsandbytesPrecision) for p in plugins) and not torch.cuda.is_available():\n    plugins = [p for p in plugins if not isinstance(p, BitsandbytesPrecision)]\ntrainer = Trainer(plugins=plugins)","typeGuard":"def bitsandbytes_usable(plugins) -> bool:\n    import torch\n    from lightning.pytorch.plugins import BitsandbytesPrecision\n    return not any(isinstance(p, BitsandbytesPrecision) for p in (plugins or [])) or torch.cuda.is_available()","tryCatchPattern":"try:\n    trainer = Trainer(plugins=plugins)\nexcept RuntimeError as e:\n    if \"Bitsandbytes\" in str(e):\n        trainer = Trainer(plugins=[p for p in plugins if \"Bitsandbytes\" not in type(p).__name__])\n    else:\n        raise","preventionTips":["Only enable quantization plugins when torch.cuda.is_available() is True","Skip bitsandbytes in CPU CI runs via a config flag"],"tags":["pytorch-lightning","bitsandbytes","quantization","cuda-only"],"backgroundTag":"cuda-only-feature","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}