unslothai/unsloth · error · ValueError

base_precision={mode!r} needs a CUDA GPU; this host has none

Error message

base_precision={mode!r} needs a CUDA GPU; this host has none. Use base_precision='nf4' or 'auto'.

What it means

Raised by _resolve_base_precision() when an explicitly configured base_precision is one of the dense CUDA-only modes (bf16, int8, fp8, mxfp8) but the resolved training device is not CUDA. The /info endpoint never advertises these modes on GPU-less hosts, so an explicit request usually comes from a stale cached client or a direct API call. The check fails fast instead of silently loading a full dense transformer onto the CPU, which would be unusably slow.

Source

Thrown at studio/backend/core/training/diffusion_dit_trainer.py:542

        components = base_repo_bf16_components_gb(base_model)
        if components:
            return float(components[0])
    except Exception:  # noqa: BLE001 -- table miss / import failure -> the family number
        pass
    return float(spec.dense_bf16_gb)


def _resolve_base_precision(cfg, spec, device) -> str:
    """Resolve "auto" against the live GPU (free VRAM measured BEFORE anything loads);
    explicit modes pass through (normalized() already validated them against the repo and
    compute dtype) but are re-checked against the live device here: the dense modes are
    CUDA-only, and /info never advertises them on a host without a GPU, so an explicit
    request from a stale or direct client fails fast instead of loading a full dense
    transformer onto the CPU."""
    mode = (cfg.base_precision or "nf4").strip().lower()
    if mode != "auto":
        if mode in ("bf16", "int8", "fp8", "mxfp8") and device != "cuda":
            raise ValueError(
                f"base_precision={mode!r} needs a CUDA GPU; this host has none. "
                f"Use base_precision='nf4' or 'auto'."
            )
        # int8 has no runtime fallback, so an explicit int8 against a missing torchao (or the Windows-ROCm stub) would leave the
        # transformer dense with compile disabled. The auto pick and /info gate on a FUNCTIONAL torchao; do the same here.
        if mode == "int8" and not has_functional_torchao():
            raise ValueError(
                "base_precision='int8' needs a functional torchao install; this host's "
                "torchao is missing or the non-functional Windows-ROCm stub. Use "
                "base_precision='nf4', 'bf16', or 'auto'."
            )
        # The stub answers torchao.float8 / torchao.prototype.mx_formats with a no-op that reports success, so the run would report fp8 while training bf16.
        # Keyed on the stub, not has_functional_torchao(): that probes int8's symbols, and a real-but-partial torchao must still reach the arch checks below.
        if mode in ("fp8", "mxfp8") and is_stubbed("torchao"):
            raise ValueError(
                f"base_precision={mode!r} is not available on this host: torchao is the "
                "non-functional Windows-ROCm stub. Use base_precision='nf4', 'bf16', or 'auto'."
            )

View on GitHub (pinned to 203007d190)

Solutions

1. Set base_precision to 'nf4' (the CPU-safe 4-bit floor) or 'auto' and restart the run.
2. If dense training is required, run on a host with a working CUDA GPU (verify torch.cuda.is_available() in the same environment).
3. Refresh the Studio client / clear cached settings so a stale config from a GPU host is not replayed on this host.

Example fix

# before
cfg.base_precision = "bf16"  # on a host with no CUDA GPU

# after
cfg.base_precision = "auto"  # resolves to nf4 without a GPU; nf4 explicitly also works

import torch
DENSE_MODES = {"bf16", "int8", "fp8", "mxfp8"}

def precision_ok(mode: str) -> bool:
    return mode.strip().lower() not in DENSE_MODES or torch.cuda.is_available()

try:
    mode = _resolve_base_precision(cfg, spec, device)
except ValueError as e:
    if "needs a CUDA GPU" in str(e):
        cfg.base_precision = "auto"
        mode = _resolve_base_precision(cfg, spec, device)
    else:
        raise

When it happens

Trigger: Setting cfg.base_precision to 'bf16', 'int8', 'fp8', or 'mxfp8' on a host where torch.cuda.is_available() is False (device resolves to 'cpu'); a stale Studio client replaying a previously saved config after the GPU was removed/drivers broken; direct API calls that skip the /info capability gate.

Common situations: Moving a config between a GPU workstation and a CPU-only box; CUDA driver update breaking torch.cuda.is_available(); Docker images built without CUDA runtime; a saved preset from a GPU host reused on a laptop.

Related errors

• base_precision={base_precision!r} trains in bf16 compute; se
• base_precision={base_precision!r} is not supported for minim
• transformer_quant='{requested}' could not be used: {reason}.
• STT model loading was cancelled so training could start.
• No DiT trainer for family {cfg.resolved_family!r}

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/5856e4d81c9decef. Report an issue: GitHub.