unslothai/unsloth · error · LlamaServerNotFoundError

This is a GGUF model, but the llama.cpp runtime (llama-serve

Error message

This is a GGUF model, but the llama.cpp runtime (llama-server) is not installed. Run `unsloth studio setup` to download the prebuilt runtime, then try again. (Advanced: set LLAMA_SERVER_PATH to an existing binary.)

What it means

On the model LOAD path, when detect_gguf_model_remote() confirms the HF repo has GGUF files, the code preflights LlamaCppBackend._find_llama_server_binary(include_denied=True) (include_denied so a transiently locked-but-present binary passes) and raises LlamaServerNotFoundError with this actionable detail before starting a multi-GB download. The same LLAMA_SERVER_NOT_FOUND_DETAIL constant is used by the backend itself (core/inference/llama_cpp.py), so the failure mode and remedy are consistent everywhere.

Source

Thrown at studio/backend/utils/models/model_config.py:3831

                    gguf_mmproj_file = mmproj_file,
                    gguf_mtp_file = mtp_file,
                    gguf_dspark_file = dspark_file,
                    gguf_dflash_file = dflash_file,
                )
        else:
            # Does the HF repo contain GGUF files?
            gguf_filename = detect_gguf_model_remote(identifier, hf_token = hf_token)
            if gguf_filename:
                # Preflight: verify the llama-server binary exists before a multi-GB download.
                # include_denied: a transiently locked binary still exists and the lock clears in time.
                from core.inference.llama_cpp import (
                    LLAMA_SERVER_NOT_FOUND_DETAIL,
                    LlamaCppBackend,
                    LlamaServerNotFoundError,
                )

                if not LlamaCppBackend._find_llama_server_binary(include_denied = True):
                    raise LlamaServerNotFoundError(LLAMA_SERVER_NOT_FOUND_DETAIL)

                # list_gguf_variants() detects vision & resolves the variant
                variants, has_vision = list_gguf_variants(identifier, hf_token = hf_token)
                variant = gguf_variant
                if variant:
                    from core.inference.llama_cpp import (
                        _gguf_files_for_variant,
                        cached_gguf_for_load,
                    )

                    # Reject before the load path unloads the resident model.
                    # Only a live, complete repo listing can prove the variant
                    # absent; without one, let the load path resolve it. The
                    # cache escape mirrors the load path's own reuse predicate.
                    try:
                        from huggingface_hub import list_repo_files
                        repo_files = list_repo_files(identifier, token = hf_token)
                    except Exception:

View on GitHub (pinned to 203007d190)

Solutions

1. Run 'unsloth studio setup' to download the prebuilt llama-server, then retry the load
2. Or set LLAMA_SERVER_PATH to an existing llama-server binary you built/installed yourself
3. Verify discovery afterward via the llama/backend status endpoints
4. In Docker, add the runtime-download step to the image build so it is not repeated per-run

Example fix

# before: load GGUF with no runtime
load_model('unsloth/Qwen3-GGUF')
# LlamaServerNotFoundError: ... llama-server is not installed ...

# after
# terminal:
#   unsloth studio setup
load_model('unsloth/Qwen3-GGUF')  # preflight passes, download proceeds

from core.inference.llama_cpp import LlamaCppBackend

def llama_runtime_ready() -> bool:
    return bool(LlamaCppBackend._find_llama_server_binary(include_denied=True))

# if the repo is GGUF and not llama_runtime_ready(): offer 'unsloth studio setup'

from core.inference.llama_cpp import LlamaServerNotFoundError
try:
    load_model(gguf_repo_id)
except LlamaServerNotFoundError as e:
    offer_run('unsloth studio setup')  # message already contains full instructions

When it happens

Trigger: Loading any remote GGUF model when no llama-server binary is discoverable: not installed via 'unsloth studio setup', LLAMA_SERVER_PATH unset or pointing at a nonexistent file, or the managed runtime directory empty (first run, failed prior install, antivirus quarantine). include_denied=True means an install-in-progress lock does NOT trigger this — only genuine absence does.

Common situations: Fresh installs where the user skipped the runtime download; Linux servers without the prebuilt fetched; a deleted managed runtime; Docker images that include models but not the llama.cpp runtime.

Related errors

• GGUF variant '{variant}' not found in {identifier}. Availabl
• '{repo_id}' is a single-file GGUF repo; load it with model_k
• Metadata unavailable while resolving GGUF variant '{variant}
• This execution artifact is outside the Recipe Studio dataset
• Execution artifacts are no longer available.

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/b56c022d2d32027a. Report an issue: GitHub.