{"record":{"id":"d7f713eae4b6df35","repo":"unslothai/unsloth","slug":"base-precision-mxfp8-needs-a-blackwell-sm100","errorCode":null,"errorMessage":"base_precision='mxfp8' needs a Blackwell (sm100+) GPU; this GPU is older. Use base_precision='bf16', 'int8', 'nf4', or 'auto'.","messagePattern":"base_precision='mxfp8' needs a Blackwell \\(sm100\\+\\) GPU; this GPU is older\\. Use base_precision='bf16', 'int8', 'nf4', or 'auto'\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/diffusion_dit_trainer.py","lineNumber":569,"sourceCode":"                \"torchao is missing or the non-functional Windows-ROCm stub. Use \"\n                \"base_precision='nf4', 'bf16', or 'auto'.\"\n            )\n        # The stub answers torchao.float8 / torchao.prototype.mx_formats with a no-op that reports success, so the run would report fp8 while training bf16.\n        # Keyed on the stub, not has_functional_torchao(): that probes int8's symbols, and a real-but-partial torchao must still reach the arch checks below.\n        if mode in (\"fp8\", \"mxfp8\") and is_stubbed(\"torchao\"):\n            raise ValueError(\n                f\"base_precision={mode!r} is not available on this host: torchao is the \"\n                \"non-functional Windows-ROCm stub. Use base_precision='nf4', 'bf16', or 'auto'.\"\n            )\n        # mxfp8 needs Blackwell (sm100+): its MX GEMM raises at the first training step, after a full dense load. Re-check here to fail fast for a stale client.\n        if mode == \"mxfp8\" and device == \"cuda\":\n            try:\n                import torch\n                blackwell = torch.cuda.get_device_capability() >= (10, 0)\n            except Exception:  # noqa: BLE001 -- probe failure -> treat as unsupported, fail fast\n                blackwell = False\n            if not blackwell:\n                raise ValueError(\n                    \"base_precision='mxfp8' needs a Blackwell (sm100+) GPU; this GPU is older. \"\n                    \"Use base_precision='bf16', 'int8', 'nf4', or 'auto'.\"\n                )\n        return mode\n    # auto may only resolve to the dense modes when the run uses bf16 compute, mirroring the normalized() rule for explicit dense modes; otherwise stay on the nf4 floor.\n    if getattr(cfg, \"mixed_precision\", \"bf16\") != \"bf16\":\n        return \"nf4\"\n    prequant = repo_is_prequantized(cfg.base_model)\n    free_gb = None\n    capability = None\n    has_fp8 = False\n    # int8 has no runtime fallback, so gate the auto pick on a FUNCTIONAL torchao: find_spec(\"torchao\") is satisfied by the Windows-ROCm stub whose quantize_ is a no-op.\n    has_torchao = has_functional_torchao()\n    if device == \"cuda\":\n        try:\n            import torch\n\n            # Windows ROCm over-reports free VRAM (#8403), which would pick a","sourceCodeStart":551,"sourceCodeEnd":587,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/diffusion_dit_trainer.py#L551-L587","documentation":"Raised by _resolve_base_precision() when base_precision='mxfp8' runs on CUDA hardware whose compute capability is below (10, 0), i.e. anything older than Blackwell. mxfp8 relies on Blackwell's MX GEMM kernels, which otherwise raise at the first training step — after a full dense transformer load, wasting minutes of setup. A failed capability probe (any exception) is also treated as unsupported to fail fast.","triggerScenarios":"Setting base_precision='mxfp8' on Hopper (H100, sm90), Ada (RTX 40xx, sm89), Ampere or older NVIDIA GPUs; torch.cuda.get_device_capability() throwing during the probe; stale clients sending mxfp8 to a pre-Blackwell host.","commonSituations":"Renting an A100/H100 node and assuming newest quantization works; a config tuned on a B200 reused on an older cluster; multi-GPU workstations where the visible device is not the Blackwell card.","solutions":["Use base_precision='bf16', 'int8', 'nf4', or 'auto' on this GPU as the message indicates.","If mxfp8 is required, move the run to a Blackwell (sm100+) GPU such as an RTX 50-series or B-series datacenter card.","Check CUDA_VISIBLE_DEVICES so the run actually lands on the intended Blackwell device."],"exampleFix":"# before\ncfg.base_precision = \"mxfp8\"  # on an H100 (sm90)\n\n# after\ncfg.base_precision = \"fp8\"  # or \"bf16\"/\"int8\"/\"auto\" on pre-Blackwell hardware","handlingStrategy":"validation","validationCode":"import torch\n\ndef mxfp8_available() -> bool:\n    if not torch.cuda.is_available():\n        return False\n    try:\n        return torch.cuda.get_device_capability() >= (10, 0)\n    except Exception:\n        return False","typeGuard":null,"tryCatchPattern":"try:\n    mode = _resolve_base_precision(cfg, spec, device)\nexcept ValueError as e:\n    if \"Blackwell\" in str(e):\n        cfg.base_precision = \"fp8\"  # or 'bf16'/'int8'/'auto'\n        mode = _resolve_base_precision(cfg, spec, device)\n    else:\n        raise","preventionTips":["Gate mxfp8 in your UI on torch.cuda.get_device_capability() >= (10, 0).","Read the advertised modes from /info per host instead of assuming newest quantization works everywhere."],"tags":["gpu","blackwell","quantization","mxfp8"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}