Lightning-AI/pytorch-lightning · error · RuntimeError

Bitsandbytes is only supported on CUDA GPUs.

Error message

Bitsandbytes is only supported on CUDA GPUs.

What it means

BitsandbytesPrecision (quantized 8/4-bit via the bitsandbytes library) only works on CUDA GPUs. If the resolved accelerator is not a CUDAAccelerator, initialization raises RuntimeError. The check runs inside _validate_precision_choice after accelerator init.

Source

Thrown at src/lightning/pytorch/trainer/connectors/accelerator_connector.py:499

                "CPU. Using `precision='bf16-mixed'` instead."
            )
            self._precision_flag = "bf16-mixed"

        if self._precision_flag in ("16-mixed", "bf16-mixed"):
            rank_zero_info(
                f"Using {'16bit' if self._precision_flag == '16-mixed' else 'bfloat16'} Automatic Mixed Precision (AMP)"
            )
            device = self._accelerator_flag if self._accelerator_flag in ("cpu", "mps") else "cuda"
            return MixedPrecision(self._precision_flag, device)

        raise RuntimeError("No precision set")

    def _validate_precision_choice(self) -> None:
        """Validate the combination of choices for precision, AMP type, and accelerator."""
        if isinstance(self._precision_plugin_flag, BitsandbytesPrecision) and not isinstance(
            self.accelerator, CUDAAccelerator
        ):
            raise RuntimeError("Bitsandbytes is only supported on CUDA GPUs.")
        mp_precision_supported = ("32-true", "bf16-mixed", "bf16-true", "16-true")
        if (
            isinstance(self._strategy_flag, ModelParallelStrategy)
            and self._precision_flag not in mp_precision_supported
        ):
            raise ValueError(
                f"The `ModelParallelStrategy` does not support `Fabric(..., precision={self._precision_flag!r})`."
                f" Choose a different precision among: {', '.join(mp_precision_supported)}."
            )

    def _lazy_init_strategy(self) -> None:
        """Lazily set missing attributes on the previously instantiated strategy."""
        self.strategy.accelerator = self.accelerator
        if self.precision_plugin:
            self.strategy.precision_plugin = self.precision_plugin
        if self.checkpoint_io:
            self.strategy.checkpoint_io = self.checkpoint_io
        if hasattr(self.strategy, "cluster_environment"):

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Run on a CUDA GPU: set accelerator='cuda' and ensure torch.cuda.is_available() is True
2. Remove the BitsandbytesPrecision plugin when CPU training is intended

Example fix

# before
trainer = Trainer(plugins=[BitsandbytesPrecision(mode="q8")], accelerator="cpu")
# after
trainer = Trainer(plugins=[BitsandbytesPrecision(mode="q8")], accelerator="cuda")

import torch
from lightning.pytorch.plugins import BitsandbytesPrecision
if any(isinstance(p, BitsandbytesPrecision) for p in plugins) and not torch.cuda.is_available():
    plugins = [p for p in plugins if not isinstance(p, BitsandbytesPrecision)]
trainer = Trainer(plugins=plugins)

def bitsandbytes_usable(plugins) -> bool:
    import torch
    from lightning.pytorch.plugins import BitsandbytesPrecision
    return not any(isinstance(p, BitsandbytesPrecision) for p in (plugins or [])) or torch.cuda.is_available()

try:
    trainer = Trainer(plugins=plugins)
except RuntimeError as e:
    if "Bitsandbytes" in str(e):
        trainer = Trainer(plugins=[p for p in plugins if "Bitsandbytes" not in type(p).__name__])
    else:
        raise

When it happens

Trigger: Trainer(plugins=[BitsandbytesPrecision(mode='q8')], accelerator='cpu') or on a machine where accelerator auto-resolves to CPU/MPS while a bitsandbytes plugin is configured.

Common situations: QLoRA / quantized LLM fine-tuning configs run on CPU-only machines or in CI; MPS Macs attempting bitsandbytes workflows.

Related errors

• {mode!r} only works with `dtype=torch.float16`, but you chos
• You are using the bitsandbytes precision plugin, but your mo
• Instantiating your model under the `init_module` context man
• str(_BITSANDBYTES_AVAILABLE)
• The strategy `{FSDPStrategy.strategy_name}` requires a GPU a

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/238fa29f7ba61750. Report an issue: GitHub.