{"record":{"id":"e51aa10ca67df2de","repo":"unslothai/unsloth","slug":"the-lora-selection-changed-but-a-quantized-int8","errorCode":null,"errorMessage":"The LoRA selection changed, but a quantized (int8/fp8) transformer bakes its adapters at load time. Reload the model with the new adapter selection.","messagePattern":"The LoRA selection changed, but a quantized \\(int8/fp8\\) transformer bakes its adapters at load time\\. Reload the model with the new adapter selection\\.","errorType":"validation","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"studio/backend/core/inference/diffusion.py","lineNumber":5132,"sourceCode":"                pipe._unsloth_loras = tuple((n, p, 0.0) for (n, p, _w) in current)\n            return\n        desired = self._resolve_lora_set(\n            specs,\n            family = getattr(state.family, \"name\", None),\n            hf_token = state.hf_token,\n            cancel = cancel,\n        )\n        if desired == current:\n            return\n        if [(n, p) for (n, p, _w) in desired] == [(n, p) for (n, p, _w) in current]:\n            # Same adapters, new weights: value-level change on the baked topology.\n            pipe.set_adapters(\n                [n for (n, _p, _w) in desired],\n                adapter_weights = [w for (_n, _p, w) in desired],\n            )\n            pipe._unsloth_loras = desired\n            return\n        raise ValueError(\n            \"The LoRA selection changed, but a quantized (int8/fp8) transformer bakes its \"\n            \"adapters at load time. Reload the model with the new adapter selection.\"\n        )\n\n    @staticmethod\n    def _reset_step_cache(pipe: Any) -> None:\n        \"\"\"Clear the transformer's stateful step cache (FBCache) before a forward.\n\n        diffusers keys FBCache residuals by cache context (\"cond\"/\"uncond\") on the\n        long-lived transformer. The context exit does NOT reset them; the end of a\n        pipeline ``__call__`` does, via ``maybe_free_model_hooks()`` -- but only when\n        the call RETURNS. A call that raised (an OOM this generate() backs off from, a\n        cancelled denoise, a failed prior request) leaves its own batch's residual on\n        the resident transformer, and the next forward's first step then compares\n        against it: a tensor-shape mismatch when the resolution/batch changed, or a\n        stale-cache reuse otherwise. The transformer-level reset entry point is\n        ``_reset_stateful_cache`` in diffusers 0.39 (``reset_stateful_hooks`` lives only\n        on the HookRegistry, so a getattr for it on the transformer is a silent no-op).","sourceCodeStart":5114,"sourceCodeEnd":5150,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/diffusion.py#L5114-L5150","documentation":"Also from `_adjust_baked_loras`: the quantized load DID bake adapters, but the generate request's adapter set differs from the baked one. Value-level changes are fine -- the same (name, path) set with new weights goes through `set_adapters` (absorbed by torch.compile guards), and disabling all is scale 0. But a different adapter set changes topology, which the baked-and-compiled quantized transformer cannot accept, so it raises a clean ValueError directing a reload.","triggerScenarios":"Load-time loras baked as [(\"lora-a\",1.0)] but generate() passes [(\"lora-b\",0.8)], or adds/removes an adapter: the (name, path) list comparison `[(n,p) for desired] == [(n,p) for current]` fails, falling to the raise. Same set with different weights does NOT raise.","commonSituations":"UI LoRA picker letting users swap adapters on a quantized load; API clients building per-request LoRA lists without consulting what was baked; session state desync between load parameters and generate parameters.","solutions":["Keep the adapter selection identical to what was baked; adjust only weights/scales at generate time.","If a different set is needed, reload the model with the new adapter selection so it bakes correctly.","Track the baked set (pipe._unsloth_loras / load response) and diff against requested specs client-side before calling."],"exampleFix":"# before: baked [lora-a], requested [lora-b] -> raises\ndiffusion.generate(prompt=\"...\", loras=[(\"lora-b\", 0.8)])\n# after: same baked adapter, new weight -- allowed\ndiffusion.generate(prompt=\"...\", loras=[(\"lora-a\", 0.6)])","handlingStrategy":"validation","validationCode":"# Diff requested adapters against the baked set before generate\nbaked = {(n, p) for (n, p, _w) in state.baked_loras}\nrequested = [(n, p) for (n, p, _w) in normalize(lora_specs)]\nif set(requested) != baked:\n    await diffusion.load(model, transformer_quant=state.transformer_quant, loras=lora_specs)  # re-bake","typeGuard":"def same_adapter_set(desired: list, current: list) -> bool:\n    \"\"\"Baked topology matches: same (name, path) pairs regardless of weights.\"\"\"\n    return [(n, p) for (n, p, _w) in desired] == [(n, p) for (n, p, _w) in current]","tryCatchPattern":"try:\n    diffusion.generate(prompt=p, loras=loras)\nexcept ValueError as e:\n    if \"selection changed\" in str(e):\n        await diffusion.load(model, transformer_quant=\"int8\", loras=loras)\n        return diffusion.generate(prompt=p)\n    raise","preventionTips":["On quantized builds, treat adapter identity as load-time immutable; only scales vary per generate.","Keep the baked set (from pipe._unsloth_loras / load response) as the source of truth in clients.","UIs should force a reload flow when the user swaps adapters on a quantized load."],"tags":["diffusion","lora","quantization","torchao","topology"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}