unslothai/unsloth · warning · HTTPException
Unload the model before deleting
Error message
Unload the model before deleting
What it means
400 from the llama.cpp guard (models.py:3031): the model is fully loaded into llama.cpp (is_loaded, matching model_identifier and, for GGUF, a variant alias-aware match via _variant_names_same_checkpoint) and the user asked to delete it. The endpoint requires an explicit unload first — it will not yank weights from under a resident inference engine, which would segfault or corrupt active sessions.
Source
Thrown at studio/backend/routes/models.py:3031
):
raise HTTPException(
status_code = 409,
detail = "Cannot delete a model while it is loading",
)
if (
llama_backend.is_loaded
and llama_backend.model_identifier
and _loaded_model_matches_deleted_path(
llama_backend.model_identifier,
target_path,
)
and (
not gguf_variant
or not llama_backend.hf_variant
or _variant_names_same_checkpoint(llama_backend.hf_variant, gguf_variant)
)
):
raise HTTPException(
status_code = 400,
detail = "Unload the model before deleting",
)
except HTTPException:
raise
except Exception as e:
logger.warning("Could not check llama.cpp loaded model before delete: %s", e)
raise HTTPException(
status_code = 503,
detail = "Could not verify model load status before deleting",
) from e
try:
# Peek: building an orchestrator to learn there is none reaches get_device() (a torch import).
from core.inference.orchestrator import peek_inference_backend
inference_backend = peek_inference_backend()
if inference_backend is not None:
loading_models = getattr(inference_backend, "loading_models", set())View on GitHub (pinned to 203007d190)
Solutions
- Unload the model in the UI/llama.cpp endpoint first, then retry the delete.
- Scripts: call unload before delete and treat 400 'Unload the model before deleting' as 'call unload then retry once'.
- Close chat sessions bound to the model so nothing re-loads it.
- Check the llama.cpp status endpoint to see which model_identifier is resident if unsure.
Example fix
# before
r = delete(payload) # 400 while model is loaded
# after
r = delete(payload)
if r.status_code == 400 and 'Unload the model' in r.json()['detail']:
unload(llama_backend_model_identifier)
r = delete(payload) Defensive patterns
Strategy: retry
Validate before calling
status = get_llama_cpp_status()
if status.get('is_loaded') and model_matches(status.get('model_identifier'), payload['model_path']):
unload_llama_cpp() # explicit unload before delete Type guard
def llama_holds_target(status: dict, target: str) -> bool:
return bool(status.get('is_loaded') and status.get('model_identifier') and paths_match(status['model_identifier'], target)) Try / catch
r = delete(payload)
if r.status_code == 400 and r.json().get('detail') == 'Unload the model before deleting':
unload_model(); r = delete(payload)
return r Prevention
- Unload models from the chat tab before cleaning them up in the models tab.
- Automate unload-then-delete in scripts instead of assuming delete implies unload.
- Watch for variant aliasing: the loaded name may differ in spelling from the delete row's name.
When it happens
Trigger: Model is open in the llama.cpp chat tab and the user hits Delete on the same row; a GGUF loaded under its bare quant name ('Q4_K_M') being deleted via the qualified key (or vice versa) — the alias-aware match still catches it, per the source comment; deleting a parent directory that contains the loaded GGUF.
Common situations: Cleanup while a chat session is still open on that model; background services pinning a default model that retention scripts try to remove; switching models but the old one stays resident in llama.cpp.
Related errors
- model_path is required
- gguf_variant is required when export_type is 'gguf'
- Model path is outside Unsloth storage
- Refusing to delete storage root
- GGUF variant deletion requires an export directory
AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15).
Data as JSON: /api/errors/aef9ed069a28a2db.
Report an issue: GitHub.