unslothai/unsloth · error · ValueError

base_precision='mxfp8' needs a Blackwell (sm100+) GPU; this

Error message

base_precision='mxfp8' needs a Blackwell (sm100+) GPU; this GPU is older. Use base_precision='bf16', 'int8', 'nf4', or 'auto'.

What it means

Raised by _resolve_base_precision() when base_precision='mxfp8' runs on CUDA hardware whose compute capability is below (10, 0), i.e. anything older than Blackwell. mxfp8 relies on Blackwell's MX GEMM kernels, which otherwise raise at the first training step — after a full dense transformer load, wasting minutes of setup. A failed capability probe (any exception) is also treated as unsupported to fail fast.

Source

Thrown at studio/backend/core/training/diffusion_dit_trainer.py:569

                "torchao is missing or the non-functional Windows-ROCm stub. Use "
                "base_precision='nf4', 'bf16', or 'auto'."
            )
        # The stub answers torchao.float8 / torchao.prototype.mx_formats with a no-op that reports success, so the run would report fp8 while training bf16.
        # Keyed on the stub, not has_functional_torchao(): that probes int8's symbols, and a real-but-partial torchao must still reach the arch checks below.
        if mode in ("fp8", "mxfp8") and is_stubbed("torchao"):
            raise ValueError(
                f"base_precision={mode!r} is not available on this host: torchao is the "
                "non-functional Windows-ROCm stub. Use base_precision='nf4', 'bf16', or 'auto'."
            )
        # mxfp8 needs Blackwell (sm100+): its MX GEMM raises at the first training step, after a full dense load. Re-check here to fail fast for a stale client.
        if mode == "mxfp8" and device == "cuda":
            try:
                import torch
                blackwell = torch.cuda.get_device_capability() >= (10, 0)
            except Exception:  # noqa: BLE001 -- probe failure -> treat as unsupported, fail fast
                blackwell = False
            if not blackwell:
                raise ValueError(
                    "base_precision='mxfp8' needs a Blackwell (sm100+) GPU; this GPU is older. "
                    "Use base_precision='bf16', 'int8', 'nf4', or 'auto'."
                )
        return mode
    # auto may only resolve to the dense modes when the run uses bf16 compute, mirroring the normalized() rule for explicit dense modes; otherwise stay on the nf4 floor.
    if getattr(cfg, "mixed_precision", "bf16") != "bf16":
        return "nf4"
    prequant = repo_is_prequantized(cfg.base_model)
    free_gb = None
    capability = None
    has_fp8 = False
    # int8 has no runtime fallback, so gate the auto pick on a FUNCTIONAL torchao: find_spec("torchao") is satisfied by the Windows-ROCm stub whose quantize_ is a no-op.
    has_torchao = has_functional_torchao()
    if device == "cuda":
        try:
            import torch

            # Windows ROCm over-reports free VRAM (#8403), which would pick a

View on GitHub (pinned to 203007d190)

Solutions

Use base_precision='bf16', 'int8', 'nf4', or 'auto' on this GPU as the message indicates.
If mxfp8 is required, move the run to a Blackwell (sm100+) GPU such as an RTX 50-series or B-series datacenter card.
Check CUDA_VISIBLE_DEVICES so the run actually lands on the intended Blackwell device.

Example fix

# before
cfg.base_precision = "mxfp8"  # on an H100 (sm90)

# after
cfg.base_precision = "fp8"  # or "bf16"/"int8"/"auto" on pre-Blackwell hardware

Defensive patterns

Strategy: validation

Validate before calling

import torch

def mxfp8_available() -> bool:
    if not torch.cuda.is_available():
        return False
    try:
        return torch.cuda.get_device_capability() >= (10, 0)
    except Exception:
        return False

Try / catch

try:
    mode = _resolve_base_precision(cfg, spec, device)
except ValueError as e:
    if "Blackwell" in str(e):
        cfg.base_precision = "fp8"  # or 'bf16'/'int8'/'auto'
        mode = _resolve_base_precision(cfg, spec, device)
    else:
        raise

Prevention

Gate mxfp8 in your UI on torch.cuda.get_device_capability() >= (10, 0).
Read the advertised modes from /info per host instead of assuming newest quantization works everywhere.

When it happens

Trigger: Setting base_precision='mxfp8' on Hopper (H100, sm90), Ada (RTX 40xx, sm89), Ampere or older NVIDIA GPUs; torch.cuda.get_device_capability() throwing during the probe; stale clients sending mxfp8 to a pre-Blackwell host.

Common situations: Renting an A100/H100 node and assuming newest quantization works; a config tuned on a B200 reused on an older cluster; multi-GPU workstations where the visible device is not the Blackwell card.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/d7f713eae4b6df35. Report an issue: GitHub.