{"record":{"id":"5856e4d81c9decef","repo":"unslothai/unsloth","slug":"base-precision-mode-r-needs-a-cuda-gpu-this-hos","errorCode":null,"errorMessage":"base_precision={mode!r} needs a CUDA GPU; this host has none. Use base_precision='nf4' or 'auto'.","messagePattern":"base_precision=(.+?) needs a CUDA GPU; this host has none\\. Use base_precision='nf4' or 'auto'\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/diffusion_dit_trainer.py","lineNumber":542,"sourceCode":"        components = base_repo_bf16_components_gb(base_model)\n        if components:\n            return float(components[0])\n    except Exception:  # noqa: BLE001 -- table miss / import failure -> the family number\n        pass\n    return float(spec.dense_bf16_gb)\n\n\ndef _resolve_base_precision(cfg, spec, device) -> str:\n    \"\"\"Resolve \"auto\" against the live GPU (free VRAM measured BEFORE anything loads);\n    explicit modes pass through (normalized() already validated them against the repo and\n    compute dtype) but are re-checked against the live device here: the dense modes are\n    CUDA-only, and /info never advertises them on a host without a GPU, so an explicit\n    request from a stale or direct client fails fast instead of loading a full dense\n    transformer onto the CPU.\"\"\"\n    mode = (cfg.base_precision or \"nf4\").strip().lower()\n    if mode != \"auto\":\n        if mode in (\"bf16\", \"int8\", \"fp8\", \"mxfp8\") and device != \"cuda\":\n            raise ValueError(\n                f\"base_precision={mode!r} needs a CUDA GPU; this host has none. \"\n                f\"Use base_precision='nf4' or 'auto'.\"\n            )\n        # int8 has no runtime fallback, so an explicit int8 against a missing torchao (or the Windows-ROCm stub) would leave the\n        # transformer dense with compile disabled. The auto pick and /info gate on a FUNCTIONAL torchao; do the same here.\n        if mode == \"int8\" and not has_functional_torchao():\n            raise ValueError(\n                \"base_precision='int8' needs a functional torchao install; this host's \"\n                \"torchao is missing or the non-functional Windows-ROCm stub. Use \"\n                \"base_precision='nf4', 'bf16', or 'auto'.\"\n            )\n        # The stub answers torchao.float8 / torchao.prototype.mx_formats with a no-op that reports success, so the run would report fp8 while training bf16.\n        # Keyed on the stub, not has_functional_torchao(): that probes int8's symbols, and a real-but-partial torchao must still reach the arch checks below.\n        if mode in (\"fp8\", \"mxfp8\") and is_stubbed(\"torchao\"):\n            raise ValueError(\n                f\"base_precision={mode!r} is not available on this host: torchao is the \"\n                \"non-functional Windows-ROCm stub. Use base_precision='nf4', 'bf16', or 'auto'.\"\n            )","sourceCodeStart":524,"sourceCodeEnd":560,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/diffusion_dit_trainer.py#L524-L560","documentation":"Raised by _resolve_base_precision() when an explicitly configured base_precision is one of the dense CUDA-only modes (bf16, int8, fp8, mxfp8) but the resolved training device is not CUDA. The /info endpoint never advertises these modes on GPU-less hosts, so an explicit request usually comes from a stale cached client or a direct API call. The check fails fast instead of silently loading a full dense transformer onto the CPU, which would be unusably slow.","triggerScenarios":"Setting cfg.base_precision to 'bf16', 'int8', 'fp8', or 'mxfp8' on a host where torch.cuda.is_available() is False (device resolves to 'cpu'); a stale Studio client replaying a previously saved config after the GPU was removed/drivers broken; direct API calls that skip the /info capability gate.","commonSituations":"Moving a config between a GPU workstation and a CPU-only box; CUDA driver update breaking torch.cuda.is_available(); Docker images built without CUDA runtime; a saved preset from a GPU host reused on a laptop.","solutions":["Set base_precision to 'nf4' (the CPU-safe 4-bit floor) or 'auto' and restart the run.","If dense training is required, run on a host with a working CUDA GPU (verify torch.cuda.is_available() in the same environment).","Refresh the Studio client / clear cached settings so a stale config from a GPU host is not replayed on this host."],"exampleFix":"# before\ncfg.base_precision = \"bf16\"  # on a host with no CUDA GPU\n\n# after\ncfg.base_precision = \"auto\"  # resolves to nf4 without a GPU; nf4 explicitly also works","handlingStrategy":"validation","validationCode":"import torch\nDENSE_MODES = {\"bf16\", \"int8\", \"fp8\", \"mxfp8\"}\n\ndef precision_ok(mode: str) -> bool:\n    return mode.strip().lower() not in DENSE_MODES or torch.cuda.is_available()","typeGuard":null,"tryCatchPattern":"try:\n    mode = _resolve_base_precision(cfg, spec, device)\nexcept ValueError as e:\n    if \"needs a CUDA GPU\" in str(e):\n        cfg.base_precision = \"auto\"\n        mode = _resolve_base_precision(cfg, spec, device)\n    else:\n        raise","preventionTips":["Prefer base_precision='auto' — it resolves against the live GPU and never trips this guard.","Verify torch.cuda.is_available() in the target env before choosing a dense mode.","Treat saved presets from GPU hosts as host-specific; re-validate on the new machine."],"tags":["config","gpu","precision","training"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}