{"record":{"id":"b24ced3b3f4526e1","repo":"unslothai/unsloth","slug":"gguf-error","errorCode":null,"errorMessage":"gguf_error","messagePattern":"gguf_error","errorType":"http","errorClass":"HTTPException","httpStatus":409,"severity":"error","filePath":"studio/backend/routes/settings.py","lineNumber":1658,"sourceCode":"\n            # Require a genuinely loadable cache (config + weights), not just a resolved refs/main,\n            # so a metadata-only partial cache still gets the forceable 409.\n            offline_cached = local_only_load and hf_cache_snapshot_is_loadable(model)\n            if not offline_cached:\n                raise HTTPException(\n                    status_code = 409,\n                    detail = (\n                        f\"Could not verify {model!r} as an embedding model on \"\n                        \"Hugging Face (it may be the wrong model type, gated, or \"\n                        \"you may be offline).\"\n                    ),\n                )\n        # The Hub GGUF probe (list_repo_files) can hang offline; skip it. Local check stays.\n        gguf_error = _local_gguf_backend_error(model)\n        if gguf_error is None and not local_only_load:\n            gguf_error = _hf_gguf_backend_error(model, hf_token)\n        if gguf_error:\n            raise HTTPException(status_code = 409, detail = gguf_error)\n    set_rag_embedding_model(model)\n    logger.info(\n        \"settings.embedding_model_updated subject=%s model=%s forced=%s\",\n        current_subject,\n        model,\n        payload.force,\n    )\n    return _embedding_model_response()\n\n\n@router.delete(\"/embedding-model\", response_model = EmbeddingModelResponse)\ndef reset_embedding_model(\n    current_subject: str = Depends(get_current_subject),\n) -> EmbeddingModelResponse:\n    \"\"\"Clear the override, returning to the env/default model.\"\"\"\n    reset_rag_embedding_model()\n    logger.info(\"settings.embedding_model_reset subject=%s\", current_subject)\n    return _embedding_model_response()","sourceCodeStart":1640,"sourceCodeEnd":1676,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/routes/settings.py#L1640-L1676","documentation":"HTTP 409 from the embedding-model endpoint when the GGUF backend probe returns an error string. gguf_error is first computed by _local_gguf_backend_error(model) (always), and if that is None and the server is not local-only-load, by _hf_gguf_backend_error(model, hf_token) — a Hub probe that is intentionally skipped offline because list_repo_files can hang. A non-empty gguf_error means the model cannot serve as the GGUF-backed embedder on the active llama-server backend; the detail is the specific probe message (missing .gguf files, wrong layout, unreachable repo, ...).","triggerScenarios":"PUT embedding-model naming a GGUF repo while the llama-server backend is active, where the repo has no usable .gguf files locally (local probe fails), or — when online — the Hub probe finds no downloadable GGUF weights or the repo is inaccessible with the supplied token.","commonSituations":"Pointing the embedder at a GGUF-quantized repo that actually ships only safetensors; partial local GGUF cache; gated GGUF repos without a token; typos in the repo id surfacing as probe errors online.","solutions":["Read the detail string — it distinguishes 'no local GGUF found' from Hub-side failures.","Verify the repo really hosts .gguf files (check the Files tab) and that at least one is fully downloaded locally.","For gated GGUF repos, set a token with access via the token endpoint and retry while online.","If the repo has no GGUF variant, either switch to a safetensors sentence-transformers model or change the backend so the GGUF path is not required."],"exampleFix":"# before\nPUT /settings/embedding-model {\"model\": \"user/mixed-model\"}  # 409: no .gguf weights\n\n# after\n# confirm GGUF exists on the Hub, e.g. user/mixed-model-GGUF\nhuggingface-cli download user/mixed-model-GGUF --include \"*.gguf\"\nPUT /settings/embedding-model {\"model\": \"user/mixed-model-GGUF\"}","handlingStrategy":"validation","validationCode":"const localGgufs = await listLocalCacheFiles(model).then(fs => fs.filter(f => f.endsWith('.gguf')));\nif (localGgufs.length === 0 && !navigator.onLine) {\n  throw new Error('No local .gguf files and offline; cannot use as GGUF embedder');\n}\nawait api.put('/settings/embedding-model', { model });","typeGuard":"function hasGgufWeights(files: string[]): boolean {\n  return files.some(f => f.toLowerCase().endsWith('.gguf'));\n}","tryCatchPattern":"try { await api.put('/settings/embedding-model', { model }); }\ncatch (e) {\n  if (e.status === 409 && /gguf/i.test(e.detail)) { showGgufHelp(e.detail); return; }\n  throw e;\n}","preventionTips":["Confirm the repo actually publishes .gguf files before naming it as the embedder.","Fully download at least one .gguf file before going offline.","Keep the HF token configured so the online Hub probe can read gated GGUF repos."],"tags":["fastapi","http-409","gguf","llama-server","embeddings","hugging-face","offline"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}