unslothai/unsloth · error · RuntimeError

Could not switch the diffusion engine to {name}: unloading t

Error message

Could not switch the diffusion engine to {name}: unloading the current {old_name} model failed ({exc}). The current model is still loaded; unload it and try again.

What it means

Switching the active diffusion engine requires unloading the current one; that unload raised. The router deliberately does NOT publish the new engine after a failed teardown, because the old model still holds VRAM and flipping the name would hide it from get_active_diffusion_engine() — making the leak permanent. The old engine stays active and reclaimable so the caller can retry.

Source

Thrown at studio/backend/core/inference/diffusion_engine_router.py:122

        old_name = None
        with _lock:
            if name != _active_engine_name:
                engine_to_unload = get_active_diffusion_engine()
                old_name = _active_engine_name
            else:
                # No engine change: publish the (possibly refreshed) fallback reason now.
                _fallback_reason = reason if name == ENGINE_DIFFUSERS else None
        if engine_to_unload is not None:
            # Publish the new engine only AFTER the old one unloads: the evictor unloads get_active_diffusion_engine(), so flipping
            # the name first would let a concurrent acquire_for evict the new (empty) engine while the old model frees VRAM.
            try:
                engine_to_unload.unload()
            except Exception as exc:
                # Do NOT publish the new engine after a failed teardown. The old model (or the resident sd-server) still holds its memory, and flipping the
                # name would hide it from get_active_diffusion_engine(), which the evictor, /images/unload and the next load all resolve through, so the leak
                # would be permanent. Leaving the old engine active keeps it reclaimable and lets the caller retry.
                logger.error("failed to unload previous engine %s: %s", old_name, exc)
                raise RuntimeError(
                    f"Could not switch the diffusion engine to {name}: unloading the current "
                    f"{old_name} model failed ({exc}). The current model is still loaded; "
                    "unload it and try again."
                ) from exc
            with _lock:
                _active_engine_name = name
                _fallback_reason = reason if name == ENGINE_DIFFUSERS else None
        if name == ENGINE_SD_CPP:
            logger.info("diffusion engine: sd_cpp")
        else:
            logger.info("diffusion engine: diffusers (%s)", reason or "selected")
        return get_active_diffusion_engine()


def begin_load_on(expected_engine: Any, start: Callable[[], Any]) -> Any:
    """Run ``start`` under the transition lock, refusing if the engine changed since selection.

    A load route selects its engine, then yields (device probe, arbiter acquire) before it

View on GitHub (pinned to 203007d190)

Solutions

1. Retry the switch after manually unloading: call the unload endpoint (/images/unload) or otherwise free the current model, then reissue the load
2. If unload keeps failing, restart the backend process to clear the wedged resident model
3. Check logs for the underlying unload exception ('failed to unload previous engine ...') and fix that root cause (kill a stuck sd-server, resolve CUDA errors)

Example fix

# before
engine = select_and_activate_engine(...)  # RuntimeError: unload failed
engine = select_and_activate_engine(...)  # blind retry, same failure

# after
try:
    engine = select_and_activate_engine(...)
except RuntimeError:
    unload_current_model()          # /images/unload or get_active_diffusion_engine().unload()
    engine = select_and_activate_engine(...)  # now the old engine is gone, switch succeeds

def engine_switch_safe() -> bool:
    # best-effort preflight: current engine can unload without CUDA/resident issues
    eng = get_active_diffusion_engine()
    try:
        return eng is None or eng.can_unload() if hasattr(eng, "can_unload") else True
    except Exception:
        return False

try:
    engine = select_and_activate_engine(fam, ...)
except RuntimeError as e:
    if "unload it and try again" in str(e):
        get_active_diffusion_engine().unload()  # or POST /images/unload
        engine = select_and_activate_engine(fam, ...)  # retry once
    else:
        raise

When it happens

Trigger: A second /images/load (or any engine switch) targeting the other engine while the current engine's unload() raises — e.g. sd-server resident process refusing to stop, CUDA errors during model free, or a diffusers pipeline teardown exception.

Common situations: Two concurrent loads racing across engines; a wedged resident sd-server process; transient CUDA/driver errors while freeing VRAM; repeated engine flip-flopping under load.

Related errors

• The diffusion engine changed while this load was starting. R
• text_encoder_quant='{requested}' could not be used: {reason}
• A diffusion load is already in progress.
• The requested LoRA adapters could not be applied: baking ada
• Diffusion generation was cancelled.

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/81a12222ea436000. Report an issue: GitHub.