{"record":{"id":"ff9bf1b44732fefa","repo":"unslothai/unsloth","slug":"base-precision-int8-needs-a-functional-torchao-i","errorCode":null,"errorMessage":"base_precision='int8' needs a functional torchao install; this host's torchao is missing or the non-functional Windows-ROCm stub. Use base_precision='nf4', 'bf16', or 'auto'.","messagePattern":"base_precision='int8' needs a functional torchao install; this host's torchao is missing or the non-functional Windows-ROCm stub\\. Use base_precision='nf4', 'bf16', or 'auto'\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/diffusion_dit_trainer.py","lineNumber":549,"sourceCode":"\ndef _resolve_base_precision(cfg, spec, device) -> str:\n    \"\"\"Resolve \"auto\" against the live GPU (free VRAM measured BEFORE anything loads);\n    explicit modes pass through (normalized() already validated them against the repo and\n    compute dtype) but are re-checked against the live device here: the dense modes are\n    CUDA-only, and /info never advertises them on a host without a GPU, so an explicit\n    request from a stale or direct client fails fast instead of loading a full dense\n    transformer onto the CPU.\"\"\"\n    mode = (cfg.base_precision or \"nf4\").strip().lower()\n    if mode != \"auto\":\n        if mode in (\"bf16\", \"int8\", \"fp8\", \"mxfp8\") and device != \"cuda\":\n            raise ValueError(\n                f\"base_precision={mode!r} needs a CUDA GPU; this host has none. \"\n                f\"Use base_precision='nf4' or 'auto'.\"\n            )\n        # int8 has no runtime fallback, so an explicit int8 against a missing torchao (or the Windows-ROCm stub) would leave the\n        # transformer dense with compile disabled. The auto pick and /info gate on a FUNCTIONAL torchao; do the same here.\n        if mode == \"int8\" and not has_functional_torchao():\n            raise ValueError(\n                \"base_precision='int8' needs a functional torchao install; this host's \"\n                \"torchao is missing or the non-functional Windows-ROCm stub. Use \"\n                \"base_precision='nf4', 'bf16', or 'auto'.\"\n            )\n        # The stub answers torchao.float8 / torchao.prototype.mx_formats with a no-op that reports success, so the run would report fp8 while training bf16.\n        # Keyed on the stub, not has_functional_torchao(): that probes int8's symbols, and a real-but-partial torchao must still reach the arch checks below.\n        if mode in (\"fp8\", \"mxfp8\") and is_stubbed(\"torchao\"):\n            raise ValueError(\n                f\"base_precision={mode!r} is not available on this host: torchao is the \"\n                \"non-functional Windows-ROCm stub. Use base_precision='nf4', 'bf16', or 'auto'.\"\n            )\n        # mxfp8 needs Blackwell (sm100+): its MX GEMM raises at the first training step, after a full dense load. Re-check here to fail fast for a stale client.\n        if mode == \"mxfp8\" and device == \"cuda\":\n            try:\n                import torch\n                blackwell = torch.cuda.get_device_capability() >= (10, 0)\n            except Exception:  # noqa: BLE001 -- probe failure -> treat as unsupported, fail fast\n                blackwell = False","sourceCodeStart":531,"sourceCodeEnd":567,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/diffusion_dit_trainer.py#L531-L567","documentation":"Raised by _resolve_base_precision() when base_precision='int8' is requested but has_functional_torchao() reports False. int8 quantization has no runtime fallback: without a working torchao the transformer would stay dense with compile disabled. A bare find_spec(\"torchao\") is not enough because the Windows-ROCm stub also satisfies it while its quantize_ is a no-op, so the probe tests for actually functional int8 symbols.","triggerScenarios":"Setting base_precision='int8' with torchao not installed, installed but broken/outdated, or replaced by the non-functional Windows-ROCm stub package; explicit int8 request on a host whose /info already hides the option.","commonSituations":"Custom Python env where torchao was never installed or was uninstalled during a dependency conflict; a torch upgrade leaving torchao ABI-incompatible; Windows+ROCm setups carrying the stub torchao.","solutions":["Install or repair a functional torchao matching your torch version (pip install -U torchao) and retry.","Switch base_precision to 'nf4', 'bf16', or 'auto' as the message suggests, since those paths do not require torchao.","On Windows-ROCm, accept that int8 is unavailable and use nf4/bf16 instead of the stub."],"exampleFix":"# before\ncfg.base_precision = \"int8\"  # torchao missing\n\n# after\n$ pip install -U torchao\ncfg.base_precision = \"int8\"  # now passes has_functional_torchao()\n# or: cfg.base_precision = \"nf4\"","handlingStrategy":"validation","validationCode":"from importlib.util import find_spec\n\ndef torchao_usable() -> bool:\n    if find_spec(\"torchao\") is None:\n        return False\n    try:\n        from torchao.quantization import quantize_\n        return callable(quantize_)\n    except Exception:\n        return False","typeGuard":null,"tryCatchPattern":"try:\n    mode = _resolve_base_precision(cfg, spec, device)\nexcept ValueError as e:\n    if \"functional torchao\" in str(e):\n        cfg.base_precision = \"bf16\"  # or 'nf4'\n        mode = _resolve_base_precision(cfg, spec, device)\n    else:\n        raise","preventionTips":["Pin torchao alongside torch in requirements so upgrades keep them compatible.","Smoke-test has_functional_torchao() in your environment setup script.","On Windows-ROCm, do not offer int8 in your UI."],"tags":["dependencies","torchao","quantization","training"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}