unslothai/unsloth · error · ValueError

base_precision={mode!r} is not available on this host: torch

Error message

base_precision={mode!r} is not available on this host: torchao is the non-functional Windows-ROCm stub. Use base_precision='nf4', 'bf16', or 'auto'.

What it means

Raised by _resolve_base_precision() when base_precision is 'fp8' or 'mxfp8' and is_stubbed("torchao") is True. The Windows-ROCm stub answers torchao.float8 / torchao.prototype.mx_formats with a no-op that reports success, so without this guard the run would claim fp8 while actually training bf16. The check is keyed on the stub specifically (not has_functional_torchao) so a real-but-partial torchao install still proceeds to the later architecture checks.

Source

Thrown at studio/backend/core/training/diffusion_dit_trainer.py:557

    mode = (cfg.base_precision or "nf4").strip().lower()
    if mode != "auto":
        if mode in ("bf16", "int8", "fp8", "mxfp8") and device != "cuda":
            raise ValueError(
                f"base_precision={mode!r} needs a CUDA GPU; this host has none. "
                f"Use base_precision='nf4' or 'auto'."
            )
        # int8 has no runtime fallback, so an explicit int8 against a missing torchao (or the Windows-ROCm stub) would leave the
        # transformer dense with compile disabled. The auto pick and /info gate on a FUNCTIONAL torchao; do the same here.
        if mode == "int8" and not has_functional_torchao():
            raise ValueError(
                "base_precision='int8' needs a functional torchao install; this host's "
                "torchao is missing or the non-functional Windows-ROCm stub. Use "
                "base_precision='nf4', 'bf16', or 'auto'."
            )
        # The stub answers torchao.float8 / torchao.prototype.mx_formats with a no-op that reports success, so the run would report fp8 while training bf16.
        # Keyed on the stub, not has_functional_torchao(): that probes int8's symbols, and a real-but-partial torchao must still reach the arch checks below.
        if mode in ("fp8", "mxfp8") and is_stubbed("torchao"):
            raise ValueError(
                f"base_precision={mode!r} is not available on this host: torchao is the "
                "non-functional Windows-ROCm stub. Use base_precision='nf4', 'bf16', or 'auto'."
            )
        # mxfp8 needs Blackwell (sm100+): its MX GEMM raises at the first training step, after a full dense load. Re-check here to fail fast for a stale client.
        if mode == "mxfp8" and device == "cuda":
            try:
                import torch
                blackwell = torch.cuda.get_device_capability() >= (10, 0)
            except Exception:  # noqa: BLE001 -- probe failure -> treat as unsupported, fail fast
                blackwell = False
            if not blackwell:
                raise ValueError(
                    "base_precision='mxfp8' needs a Blackwell (sm100+) GPU; this GPU is older. "
                    "Use base_precision='bf16', 'int8', 'nf4', or 'auto'."
                )
        return mode
    # auto may only resolve to the dense modes when the run uses bf16 compute, mirroring the normalized() rule for explicit dense modes; otherwise stay on the nf4 floor.
    if getattr(cfg, "mixed_precision", "bf16") != "bf16":

View on GitHub (pinned to 203007d190)

Solutions

  1. Switch base_precision to 'nf4', 'bf16', or 'auto', which are the modes actually available on this host.
  2. If you need real fp8/mxfp8, run on a supported CUDA host with a genuine torchao install (and for mxfp8, a Blackwell GPU).
  3. Uninstall the stub and install a genuine torchao if one exists for your platform, then retry.

Example fix

# before
cfg.base_precision = "fp8"  # torchao is the Windows-ROCm stub

# after
cfg.base_precision = "bf16"  # functional on this host without torchao
Defensive patterns

Strategy: validation

Validate before calling

from importlib.util import find_spec

def torchao_stubbed() -> bool:
    spec = find_spec("torchao")
    return spec is None or spec.origin is None or "stub" in (spec.origin or "")

Try / catch

try:
    mode = _resolve_base_precision(cfg, spec, device)
except ValueError as e:
    if "Windows-ROCm stub" in str(e):
        cfg.base_precision = "bf16"
        mode = _resolve_base_precision(cfg, spec, device)
    else:
        raise

Prevention

When it happens

Trigger: Explicitly setting base_precision='fp8' or 'mxfp8' on a Windows-ROCm host whose torchao is the stub package; a stale or direct client sending an fp8/mxfp8 request that /info would never advertise on such a host.

Common situations: Windows + AMD ROCm environments that ship the no-op torchao stub; reinstalling torch over a stubbed environment; configs exported from an NVIDIA host reused on a ROCm box.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/81fc1068fa6cd4d4. Report an issue: GitHub.