unslothai/unsloth · error · ValueError

{spec.family} LoRA training requires bf16: fp16 overflows it

Error message

{spec.family} LoRA training requires bf16: fp16 overflows its fp32 RoPE / embedder internals. Set mixed precision to bf16.

What it means

Raised when cfg.mixed_precision is 'fp16' for a DiT family whose spec sets force_bf16. These flow-matching DiT models keep fp32 RoPE and embedder internals that overflow in fp16, so the trainer refuses rather than silently upgrading the request. It is validated before the heavy diffusers imports so even hosts without diffusers installed see the real error.

Source

Thrown at studio/backend/core/training/diffusion_dit_trainer.py:1842

    config: DiffusionLoraConfig,
    *,
    on_event: Optional[EventCb] = None,
    should_stop: Optional[StopCb] = None,
) -> str:
    """Train a flow-matching DiT LoRA (FLUX.1 / FLUX.2 / Qwen-Image / Z-Image / Krea 2 / LTX-2) and export it.

    Resumable: ``cfg.resume_from_checkpoint`` restores the adapter, optimizer moments, LR
    position, EMA shadow, sampler cycle and RNG streams from a ``checkpoint-<N>`` bundle,
    and the loop runs steps N+1..train_steps (the TARGET TOTAL). A stop-and-save and every
    ``cfg.save_steps`` interval write such a bundle."""
    cfg = config.normalized()
    spec = _SPECS.get(cfg.resolved_family)
    if spec is None:
        raise ValueError(f"No DiT trainer for family {cfg.resolved_family!r}")

    # DiT families train in bf16, so an explicit fp16 request is refused, not silently upgraded. Validated before the heavy imports so a host without diffusers still sees the real error.
    if cfg.mixed_precision == "fp16" and spec.force_bf16:
        raise ValueError(
            f"{spec.family} LoRA training requires bf16: fp16 overflows its fp32 RoPE / "
            f"embedder internals. Set mixed precision to bf16."
        )

    import torch

    rng = random.Random(cfg.seed)
    torch.manual_seed(cfg.seed)
    _FLUX_STATIC.clear()

    save_on_stop = True

    def _check_stop() -> bool:
        nonlocal save_on_stop
        if should_stop is None:
            return False
        sig = should_stop()
        if not sig:

View on GitHub (pinned to 203007d190)

Solutions

1. Set mixed precision to bf16 in the run config and restart.
2. If VRAM is the concern, lower resolution or use nf4 base_precision rather than fp16.
3. Update saved presets that carry mixed_precision='fp16' so future DiT runs default to bf16.

Example fix

# before
cfg.mixed_precision = "fp16"  # FLUX LoRA run -> ValueError

# after
cfg.mixed_precision = "bf16"

def precision_valid(mixed_precision: str, force_bf16: bool) -> bool:
    return not (mixed_precision == "fp16" and force_bf16)

try:
    train_diT_lora(cfg)
except ValueError as e:
    if "requires bf16" in str(e):
        cfg.mixed_precision = "bf16"
        cfg = config.normalized()
        train_diT_lora(cfg)
    else:
        raise

When it happens

Trigger: Setting mixed_precision='fp16' while training FLUX.1 / FLUX.2 / Qwen-Image / Z-Image / Krea 2 / LTX-2 LoRAs; reusing an SDXL fp16 config verbatim for a DiT run; a UI preset that still records fp16.

Common situations: Habits carried over from SD1.5/SDXL training where fp16 was standard; older tutorials or presets; attempting fp16 to save VRAM on a pre-Ampere card (which instead hits the bf16 capability error).

Related errors

• base_precision={mode!r} needs a CUDA GPU; this host has none
• No DiT trainer for family {cfg.resolved_family!r}
• mixed_precision must be one of bf16 / fp16 / no
• '{resolved_family}' LoRA training requires bf16: fp16 overfl
• base_precision={base_precision!r} trains in bf16 compute; se

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/196cdfaca5e52880. Report an issue: GitHub.