{"record":{"id":"b56c022d2d32027a","repo":"unslothai/unsloth","slug":"this-is-a-gguf-model-but-the-llama-cpp-runtime-l","errorCode":null,"errorMessage":"This is a GGUF model, but the llama.cpp runtime (llama-server) is not installed. Run `unsloth studio setup` to download the prebuilt runtime, then try again. (Advanced: set LLAMA_SERVER_PATH to an existing binary.)","messagePattern":"This is a GGUF model, but the llama\\.cpp runtime \\(llama-server\\) is not installed\\. Run `unsloth studio setup` to download the prebuilt runtime, then try again\\. \\(Advanced: set LLAMA_SERVER_PATH to an existing binary\\.\\)","errorType":"exception","errorClass":"LlamaServerNotFoundError","httpStatus":400,"severity":"error","filePath":"studio/backend/utils/models/model_config.py","lineNumber":3831,"sourceCode":"                    gguf_mmproj_file = mmproj_file,\n                    gguf_mtp_file = mtp_file,\n                    gguf_dspark_file = dspark_file,\n                    gguf_dflash_file = dflash_file,\n                )\n        else:\n            # Does the HF repo contain GGUF files?\n            gguf_filename = detect_gguf_model_remote(identifier, hf_token = hf_token)\n            if gguf_filename:\n                # Preflight: verify the llama-server binary exists before a multi-GB download.\n                # include_denied: a transiently locked binary still exists and the lock clears in time.\n                from core.inference.llama_cpp import (\n                    LLAMA_SERVER_NOT_FOUND_DETAIL,\n                    LlamaCppBackend,\n                    LlamaServerNotFoundError,\n                )\n\n                if not LlamaCppBackend._find_llama_server_binary(include_denied = True):\n                    raise LlamaServerNotFoundError(LLAMA_SERVER_NOT_FOUND_DETAIL)\n\n                # list_gguf_variants() detects vision & resolves the variant\n                variants, has_vision = list_gguf_variants(identifier, hf_token = hf_token)\n                variant = gguf_variant\n                if variant:\n                    from core.inference.llama_cpp import (\n                        _gguf_files_for_variant,\n                        cached_gguf_for_load,\n                    )\n\n                    # Reject before the load path unloads the resident model.\n                    # Only a live, complete repo listing can prove the variant\n                    # absent; without one, let the load path resolve it. The\n                    # cache escape mirrors the load path's own reuse predicate.\n                    try:\n                        from huggingface_hub import list_repo_files\n                        repo_files = list_repo_files(identifier, token = hf_token)\n                    except Exception:","sourceCodeStart":3813,"sourceCodeEnd":3849,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/utils/models/model_config.py#L3813-L3849","documentation":"On the model LOAD path, when detect_gguf_model_remote() confirms the HF repo has GGUF files, the code preflights LlamaCppBackend._find_llama_server_binary(include_denied=True) (include_denied so a transiently locked-but-present binary passes) and raises LlamaServerNotFoundError with this actionable detail before starting a multi-GB download. The same LLAMA_SERVER_NOT_FOUND_DETAIL constant is used by the backend itself (core/inference/llama_cpp.py), so the failure mode and remedy are consistent everywhere.","triggerScenarios":"Loading any remote GGUF model when no llama-server binary is discoverable: not installed via 'unsloth studio setup', LLAMA_SERVER_PATH unset or pointing at a nonexistent file, or the managed runtime directory empty (first run, failed prior install, antivirus quarantine). include_denied=True means an install-in-progress lock does NOT trigger this — only genuine absence does.","commonSituations":"Fresh installs where the user skipped the runtime download; Linux servers without the prebuilt fetched; a deleted managed runtime; Docker images that include models but not the llama.cpp runtime.","solutions":["Run 'unsloth studio setup' to download the prebuilt llama-server, then retry the load","Or set LLAMA_SERVER_PATH to an existing llama-server binary you built/installed yourself","Verify discovery afterward via the llama/backend status endpoints","In Docker, add the runtime-download step to the image build so it is not repeated per-run"],"exampleFix":"# before: load GGUF with no runtime\nload_model('unsloth/Qwen3-GGUF')\n# LlamaServerNotFoundError: ... llama-server is not installed ...\n\n# after\n# terminal:\n#   unsloth studio setup\nload_model('unsloth/Qwen3-GGUF')  # preflight passes, download proceeds","handlingStrategy":"validation","validationCode":"from core.inference.llama_cpp import LlamaCppBackend\n\ndef llama_runtime_ready() -> bool:\n    return bool(LlamaCppBackend._find_llama_server_binary(include_denied=True))\n\n# if the repo is GGUF and not llama_runtime_ready(): offer 'unsloth studio setup'","typeGuard":null,"tryCatchPattern":"from core.inference.llama_cpp import LlamaServerNotFoundError\ntry:\n    load_model(gguf_repo_id)\nexcept LlamaServerNotFoundError as e:\n    offer_run('unsloth studio setup')  # message already contains full instructions","preventionTips":["Run 'unsloth studio setup' as part of deployment/image build when GGUF models are used","Set LLAMA_SERVER_PATH in environments with a preinstalled binary","Gate the model-load UI on runtime presence for GGUF selections"],"tags":["gguf","llama-cpp","runtime-missing","preflight","huggingface"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}