{"record":{"id":"05d29304a098538f","repo":"unslothai/unsloth","slug":"this-trainer-requires-a-bfloat16-capable-gpu-ampe","errorCode":null,"errorMessage":"This trainer requires a bfloat16-capable GPU (Ampere or newer); this CUDA device does not support bf16.","messagePattern":"This trainer requires a bfloat16-capable GPU \\(Ampere or newer\\); this CUDA device does not support bf16\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/diffusion_dit_trainer.py","lineNumber":1870,"sourceCode":"\n    save_on_stop = True\n\n    def _check_stop() -> bool:\n        nonlocal save_on_stop\n        if should_stop is None:\n            return False\n        sig = should_stop()\n        if not sig:\n            return False\n        if isinstance(sig, dict) and sig.get(\"save\") is False:\n            save_on_stop = False\n        return True\n\n    device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n    # The flow-matching + 4-bit path is bf16 throughout (fp32 on a CPU-only box, to keep import/unit tests architecture-agnostic).\n    # Fail fast on pre-Ampere CUDA, gating on NATIVE bf16 (capability major >= 8), since is_bf16_supported() counts emulation.\n    if device == \"cuda\" and not native_bf16_supported():\n        raise ValueError(\n            \"This trainer requires a bfloat16-capable GPU (Ampere or newer); \"\n            \"this CUDA device does not support bf16.\"\n        )\n    weight_dtype = torch.bfloat16 if device == \"cuda\" else torch.float32\n\n    _assert_trusted_base_model(cfg.base_model)\n    # The repo this run will FETCH, which is what the start route preflights. Checking the\n    # canonical id instead would raise here for a gated base that normalization already\n    # redirected to its ungated mirror -- after the route had answered 200 and freed the\n    # resident models, so the request fails as a dead job rather than as a fast 400.\n    _assert_gated_access(cfg.fetch_base_model or cfg.base_model, cfg.hf_token)\n    pairs = discover_image_caption_pairs(\n        cfg.data_dir, instance_prompt = cfg.instance_prompt, caption_column = cfg.caption_column\n    )\n    # Resolve num_epochs into a concrete train_steps now the dataset size is known, and rebind cfg so every downstream read agrees.\n    cfg = replace(cfg, train_steps = resolve_train_steps(cfg, len(pairs)), num_epochs = 0)\n    # Validate a resume request against this run's identity BEFORE the multi-GB phased load, so\n    # a mismatched checkpoint fails in seconds. The identity uses the RESOLVED LoRA targets,","sourceCodeStart":1852,"sourceCodeEnd":1888,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/diffusion_dit_trainer.py#L1852-L1888","documentation":"Raised when the run resolves to a CUDA device that lacks native bf16 support, i.e. compute capability major < 8 (pre-Ampere). The whole flow-matching + 4-bit path is bf16 on GPU, and the check deliberately uses native_bf16_supported() rather than is_bf16_supported() because the latter counts emulation. CPU-only hosts are exempt — they run fp32 so import/unit tests stay architecture-agnostic.","triggerScenarios":"Training on GTX 10xx/16xx, Titan V, or other Turing/Pascal/Volta cards where device=='cuda' but capability < (8, 0); is_bf16_supported() lying via emulation on such cards.","commonSituations":"Older gaming rigs (GTX 1080, RTX 20-series); T4 instances on cheap cloud tiers (Turing, no native bf16); VMs passthrough-assigning an old GPU; free Colab/Kaggle legacy GPU runtimes.","solutions":["Move the run to an Ampere-or-newer GPU (RTX 30xx/40xx/50xx, A-series, H-series).","If stuck on this host, run CPU-only (device resolves to 'cpu' and trains fp32) — practical only for smoke tests, not real training.","Confirm the visible device: check nvidia-smi and CUDA_VISIBLE_DEVICES so an older secondary GPU is not being picked."],"exampleFix":"# before\n# training on a GTX 1080 Ti (sm61) -> ValueError on startup\n\n# after\nCUDA_VISIBLE_DEVICES=1 python train.py  # point at the RTX 3090 (sm86) instead","handlingStrategy":"validation","validationCode":"import torch\n\ndef native_bf16_gpu() -> bool:\n    if not torch.cuda.is_available():\n        return False\n    try:\n        return torch.cuda.get_device_capability()[0] >= 8\n    except Exception:\n        return False","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check compute capability (>= 8.0) before scheduling DiT training on a GPU host.","Use nvidia-smi / CUDA_VISIBLE_DEVICES to pin the run to an Ampere+ device on mixed rigs."],"tags":["gpu","bf16","hardware","training"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}