unslothai/unsloth · error · ValueError

base_precision='int8' needs a functional torchao install; th

Error message

base_precision='int8' needs a functional torchao install; this host's torchao is missing or the non-functional Windows-ROCm stub. Use base_precision='nf4', 'bf16', or 'auto'.

What it means

Raised by _resolve_base_precision() when base_precision='int8' is requested but has_functional_torchao() reports False. int8 quantization has no runtime fallback: without a working torchao the transformer would stay dense with compile disabled. A bare find_spec("torchao") is not enough because the Windows-ROCm stub also satisfies it while its quantize_ is a no-op, so the probe tests for actually functional int8 symbols.

Source

Thrown at studio/backend/core/training/diffusion_dit_trainer.py:549

def _resolve_base_precision(cfg, spec, device) -> str:
    """Resolve "auto" against the live GPU (free VRAM measured BEFORE anything loads);
    explicit modes pass through (normalized() already validated them against the repo and
    compute dtype) but are re-checked against the live device here: the dense modes are
    CUDA-only, and /info never advertises them on a host without a GPU, so an explicit
    request from a stale or direct client fails fast instead of loading a full dense
    transformer onto the CPU."""
    mode = (cfg.base_precision or "nf4").strip().lower()
    if mode != "auto":
        if mode in ("bf16", "int8", "fp8", "mxfp8") and device != "cuda":
            raise ValueError(
                f"base_precision={mode!r} needs a CUDA GPU; this host has none. "
                f"Use base_precision='nf4' or 'auto'."
            )
        # int8 has no runtime fallback, so an explicit int8 against a missing torchao (or the Windows-ROCm stub) would leave the
        # transformer dense with compile disabled. The auto pick and /info gate on a FUNCTIONAL torchao; do the same here.
        if mode == "int8" and not has_functional_torchao():
            raise ValueError(
                "base_precision='int8' needs a functional torchao install; this host's "
                "torchao is missing or the non-functional Windows-ROCm stub. Use "
                "base_precision='nf4', 'bf16', or 'auto'."
            )
        # The stub answers torchao.float8 / torchao.prototype.mx_formats with a no-op that reports success, so the run would report fp8 while training bf16.
        # Keyed on the stub, not has_functional_torchao(): that probes int8's symbols, and a real-but-partial torchao must still reach the arch checks below.
        if mode in ("fp8", "mxfp8") and is_stubbed("torchao"):
            raise ValueError(
                f"base_precision={mode!r} is not available on this host: torchao is the "
                "non-functional Windows-ROCm stub. Use base_precision='nf4', 'bf16', or 'auto'."
            )
        # mxfp8 needs Blackwell (sm100+): its MX GEMM raises at the first training step, after a full dense load. Re-check here to fail fast for a stale client.
        if mode == "mxfp8" and device == "cuda":
            try:
                import torch
                blackwell = torch.cuda.get_device_capability() >= (10, 0)
            except Exception:  # noqa: BLE001 -- probe failure -> treat as unsupported, fail fast
                blackwell = False

View on GitHub (pinned to 203007d190)

Solutions

1. Install or repair a functional torchao matching your torch version (pip install -U torchao) and retry.
2. Switch base_precision to 'nf4', 'bf16', or 'auto' as the message suggests, since those paths do not require torchao.
3. On Windows-ROCm, accept that int8 is unavailable and use nf4/bf16 instead of the stub.

Example fix

# before
cfg.base_precision = "int8"  # torchao missing

# after
$ pip install -U torchao
cfg.base_precision = "int8"  # now passes has_functional_torchao()
# or: cfg.base_precision = "nf4"

from importlib.util import find_spec

def torchao_usable() -> bool:
    if find_spec("torchao") is None:
        return False
    try:
        from torchao.quantization import quantize_
        return callable(quantize_)
    except Exception:
        return False

try:
    mode = _resolve_base_precision(cfg, spec, device)
except ValueError as e:
    if "functional torchao" in str(e):
        cfg.base_precision = "bf16"  # or 'nf4'
        mode = _resolve_base_precision(cfg, spec, device)
    else:
        raise

When it happens

Trigger: Setting base_precision='int8' with torchao not installed, installed but broken/outdated, or replaced by the non-functional Windows-ROCm stub package; explicit int8 request on a host whose /info already hides the option.

Common situations: Custom Python env where torchao was never installed or was uninstalled during a dependency conflict; a torch upgrade leaving torchao ABI-incompatible; Windows+ROCm setups carrying the stub torchao.

Related errors

• transformer_quant='{requested}' could not be used: {reason}.
• text_encoder_quant='{requested}' could not be used: {reason}
• This quantized (int8/fp8) load was built without LoRA adapte
• The LoRA selection changed, but a quantized (int8/fp8) trans
• base_precision={mode!r} is not available on this host: torch

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/ff9bf1b44732fefa. Report an issue: GitHub.