{"record":{"id":"196cdfaca5e52880","repo":"unslothai/unsloth","slug":"spec-family-lora-training-requires-bf16-fp16-ov","errorCode":null,"errorMessage":"{spec.family} LoRA training requires bf16: fp16 overflows its fp32 RoPE / embedder internals. Set mixed precision to bf16.","messagePattern":"(.+?) LoRA training requires bf16: fp16 overflows its fp32 RoPE / embedder internals\\. Set mixed precision to bf16\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/diffusion_dit_trainer.py","lineNumber":1842,"sourceCode":"    config: DiffusionLoraConfig,\n    *,\n    on_event: Optional[EventCb] = None,\n    should_stop: Optional[StopCb] = None,\n) -> str:\n    \"\"\"Train a flow-matching DiT LoRA (FLUX.1 / FLUX.2 / Qwen-Image / Z-Image / Krea 2 / LTX-2) and export it.\n\n    Resumable: ``cfg.resume_from_checkpoint`` restores the adapter, optimizer moments, LR\n    position, EMA shadow, sampler cycle and RNG streams from a ``checkpoint-<N>`` bundle,\n    and the loop runs steps N+1..train_steps (the TARGET TOTAL). A stop-and-save and every\n    ``cfg.save_steps`` interval write such a bundle.\"\"\"\n    cfg = config.normalized()\n    spec = _SPECS.get(cfg.resolved_family)\n    if spec is None:\n        raise ValueError(f\"No DiT trainer for family {cfg.resolved_family!r}\")\n\n    # DiT families train in bf16, so an explicit fp16 request is refused, not silently upgraded. Validated before the heavy imports so a host without diffusers still sees the real error.\n    if cfg.mixed_precision == \"fp16\" and spec.force_bf16:\n        raise ValueError(\n            f\"{spec.family} LoRA training requires bf16: fp16 overflows its fp32 RoPE / \"\n            f\"embedder internals. Set mixed precision to bf16.\"\n        )\n\n    import torch\n\n    rng = random.Random(cfg.seed)\n    torch.manual_seed(cfg.seed)\n    _FLUX_STATIC.clear()\n\n    save_on_stop = True\n\n    def _check_stop() -> bool:\n        nonlocal save_on_stop\n        if should_stop is None:\n            return False\n        sig = should_stop()\n        if not sig:","sourceCodeStart":1824,"sourceCodeEnd":1860,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/diffusion_dit_trainer.py#L1824-L1860","documentation":"Raised when cfg.mixed_precision is 'fp16' for a DiT family whose spec sets force_bf16. These flow-matching DiT models keep fp32 RoPE and embedder internals that overflow in fp16, so the trainer refuses rather than silently upgrading the request. It is validated before the heavy diffusers imports so even hosts without diffusers installed see the real error.","triggerScenarios":"Setting mixed_precision='fp16' while training FLUX.1 / FLUX.2 / Qwen-Image / Z-Image / Krea 2 / LTX-2 LoRAs; reusing an SDXL fp16 config verbatim for a DiT run; a UI preset that still records fp16.","commonSituations":"Habits carried over from SD1.5/SDXL training where fp16 was standard; older tutorials or presets; attempting fp16 to save VRAM on a pre-Ampere card (which instead hits the bf16 capability error).","solutions":["Set mixed precision to bf16 in the run config and restart.","If VRAM is the concern, lower resolution or use nf4 base_precision rather than fp16.","Update saved presets that carry mixed_precision='fp16' so future DiT runs default to bf16."],"exampleFix":"# before\ncfg.mixed_precision = \"fp16\"  # FLUX LoRA run -> ValueError\n\n# after\ncfg.mixed_precision = \"bf16\"","handlingStrategy":"validation","validationCode":"def precision_valid(mixed_precision: str, force_bf16: bool) -> bool:\n    return not (mixed_precision == \"fp16\" and force_bf16)","typeGuard":null,"tryCatchPattern":"try:\n    train_diT_lora(cfg)\nexcept ValueError as e:\n    if \"requires bf16\" in str(e):\n        cfg.mixed_precision = \"bf16\"\n        cfg = config.normalized()\n        train_diT_lora(cfg)\n    else:\n        raise","preventionTips":["Default new DiT configs to bf16; never copy fp16 from SDXL presets.","Validate mixed_precision against the family spec at config-build time in your UI."],"tags":["config","mixed-precision","flux","training"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}