unslothai/unsloth · error · ValueError

This quantized (int8/fp8) load was built without LoRA adapte

Error message

This quantized (int8/fp8) load was built without LoRA adapters. Reload the model with the adapter selection to bake it into the quantized transformer.

What it means

Raised by `_adjust_baked_loras` for a torchao-quantized (int8/fp8) pipeline: on such builds, adapters are baked into the transformer at load time, BEFORE quantize_ + compile, making module topology immutable at generation time. If the load had no adapters baked (`quant_baked` False) but the generate request supplies specs, adding them now is impossible, so a clean ValueError tells the client to reload with the adapter selection. Weight tweaks on the baked set and full disable (scale 0) remain allowed without reload.

Source

Thrown at studio/backend/core/inference/diffusion.py:5105

        pipe: Any,
        specs: list[tuple[str, float]],
        current: tuple,
        quant_baked: bool,
        cancel: threading.Event,
    ) -> None:
        """Generation-time LoRA handling for a torchao-quantized pipe.

        The adapters (if any) were baked at load time, before quantize_ + compile, so the
        module topology is immutable here. Allowed without a reload: weight tweaks on the
        baked set and disabling everything (scale 0 reproduces the quantized base exactly;
        set_adapters is value-level, so torch.compile guards absorb it). Anything that would
        change topology (adding adapters to a bake-less load, or a different adapter set)
        raises a clean 400 telling the client to reload with the new selection.
        """
        if not quant_baked:
            if not specs:
                return  # no adapters baked, none requested
            raise ValueError(
                "This quantized (int8/fp8) load was built without LoRA adapters. Reload the "
                "model with the adapter selection to bake it into the quantized transformer."
            )
        if not specs:
            # Disable every baked adapter: scale 0 reproduces the quantized base exactly.
            names = [n for (n, _p, _w) in current]
            if any(w != 0 for (_n, _p, w) in current):
                pipe.set_adapters(names, adapter_weights = [0.0] * len(names))
                pipe._unsloth_loras = tuple((n, p, 0.0) for (n, p, _w) in current)
            return
        desired = self._resolve_lora_set(
            specs,
            family = getattr(state.family, "name", None),
            hf_token = state.hf_token,
            cancel = cancel,
        )
        if desired == current:
            return

View on GitHub (pinned to 203007d190)

Solutions

Reload the model with the adapter selection included in the load call so it bakes into the quantized transformer.
Or reload without transformer_quant (bf16/bnb-4bit, eager speed) if per-generation LoRA hot-swap is required.
Keep the LoRA list identical between load and generate in orchestrating code.

Example fix

# before
diffusion.load(model=repo, transformer_quant="int8")
diffusion.generate(prompt="...", loras=[("my-lora", 1.0)])  # raises
# after
diffusion.load(model=repo, transformer_quant="int8", loras=[("my-lora", 1.0)])

Defensive patterns

Strategy: validation

Validate before calling

# Quantized loads are bake-only: pass loras at load time
load_kwargs = {"transformer_quant": "int8"}
if session_loras:
    load_kwargs["loras"] = session_loras  # bake now, not later
await diffusion.load(model, **load_kwargs)

Type guard

def can_hotswap_lora(state) -> bool:
    """Non-quantized (or bake-less) loads can add adapters at generate time."""
    return state.transformer_quant is None

Try / catch

try:
    diffusion.generate(prompt=p, loras=loras)
except ValueError as e:
    if "built without LoRA adapters" in str(e):
        await diffusion.load(model, transformer_quant="int8", loras=loras)  # reload + bake
        return diffusion.generate(prompt=p)
    raise

Prevention

For quantized loads, make the LoRA selection part of the load request.
Cache the load-time LoRA set in session state and reuse it on every generate.
Only weight tweaks and full-disable are generation-time operations on baked builds.

When it happens

Trigger: Loading with transformer_quant int8/fp8 but an empty loras list, then calling generate() with a non-empty loras argument: `quant_baked` is False, `specs` is non-empty, so the raise fires.

Common situations: Users loading a quantized model 'clean' for speed, then toggling a LoRA on in the UI mid-session; API clients that add LoRAs per-request assuming hot-swap; workflows that separated load config from generate config.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/73ed4b55c068443d. Report an issue: GitHub.