{"record":{"id":"aef9ed069a28a2db","repo":"unslothai/unsloth","slug":"unload-the-model-before-deleting","errorCode":null,"errorMessage":"Unload the model before deleting","messagePattern":"Unload the model before deleting","errorType":"http","errorClass":"HTTPException","httpStatus":400,"severity":"warning","filePath":"studio/backend/routes/models.py","lineNumber":3031,"sourceCode":"        ):\n            raise HTTPException(\n                status_code = 409,\n                detail = \"Cannot delete a model while it is loading\",\n            )\n        if (\n            llama_backend.is_loaded\n            and llama_backend.model_identifier\n            and _loaded_model_matches_deleted_path(\n                llama_backend.model_identifier,\n                target_path,\n            )\n            and (\n                not gguf_variant\n                or not llama_backend.hf_variant\n                or _variant_names_same_checkpoint(llama_backend.hf_variant, gguf_variant)\n            )\n        ):\n            raise HTTPException(\n                status_code = 400,\n                detail = \"Unload the model before deleting\",\n            )\n    except HTTPException:\n        raise\n    except Exception as e:\n        logger.warning(\"Could not check llama.cpp loaded model before delete: %s\", e)\n        raise HTTPException(\n            status_code = 503,\n            detail = \"Could not verify model load status before deleting\",\n        ) from e\n\n    try:\n        # Peek: building an orchestrator to learn there is none reaches get_device() (a torch import).\n        from core.inference.orchestrator import peek_inference_backend\n        inference_backend = peek_inference_backend()\n        if inference_backend is not None:\n            loading_models = getattr(inference_backend, \"loading_models\", set())","sourceCodeStart":3013,"sourceCodeEnd":3049,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/routes/models.py#L3013-L3049","documentation":"400 from the llama.cpp guard (models.py:3031): the model is fully loaded into llama.cpp (is_loaded, matching model_identifier and, for GGUF, a variant alias-aware match via _variant_names_same_checkpoint) and the user asked to delete it. The endpoint requires an explicit unload first — it will not yank weights from under a resident inference engine, which would segfault or corrupt active sessions.","triggerScenarios":"Model is open in the llama.cpp chat tab and the user hits Delete on the same row; a GGUF loaded under its bare quant name ('Q4_K_M') being deleted via the qualified key (or vice versa) — the alias-aware match still catches it, per the source comment; deleting a parent directory that contains the loaded GGUF.","commonSituations":"Cleanup while a chat session is still open on that model; background services pinning a default model that retention scripts try to remove; switching models but the old one stays resident in llama.cpp.","solutions":["Unload the model in the UI/llama.cpp endpoint first, then retry the delete.","Scripts: call unload before delete and treat 400 'Unload the model before deleting' as 'call unload then retry once'.","Close chat sessions bound to the model so nothing re-loads it.","Check the llama.cpp status endpoint to see which model_identifier is resident if unsure."],"exampleFix":"# before\nr = delete(payload)  # 400 while model is loaded\n# after\nr = delete(payload)\nif r.status_code == 400 and 'Unload the model' in r.json()['detail']:\n    unload(llama_backend_model_identifier)\n    r = delete(payload)","handlingStrategy":"retry","validationCode":"status = get_llama_cpp_status()\nif status.get('is_loaded') and model_matches(status.get('model_identifier'), payload['model_path']):\n    unload_llama_cpp()  # explicit unload before delete","typeGuard":"def llama_holds_target(status: dict, target: str) -> bool:\n    return bool(status.get('is_loaded') and status.get('model_identifier') and paths_match(status['model_identifier'], target))","tryCatchPattern":"r = delete(payload)\nif r.status_code == 400 and r.json().get('detail') == 'Unload the model before deleting':\n    unload_model(); r = delete(payload)\nreturn r","preventionTips":["Unload models from the chat tab before cleaning them up in the models tab.","Automate unload-then-delete in scripts instead of assuming delete implies unload.","Watch for variant aliasing: the loaded name may differ in spelling from the delete row's name."],"tags":["fastapi","unsloth","llama-cpp","inference","state-conflict","model-deletion"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}