unslothai/unsloth · critical · RuntimeError

llama-server embedder failed to become healthy. Last output:

Error message

llama-server embedder failed to become healthy. Last output:
{tail[:2000]}

What it means

Raised when the llama-server embedding subprocess fails its health check within EMBED_STARTUP_TIMEOUT_S after being spawned. The backend starts a llama-server process, drains its stdout in a daemon thread, polls /health, and if the server never becomes ready it kills the process and re-raises with the last 30 stdout lines (capped at 2000 chars) as diagnostics. Typical root causes are a corrupt/unsupported GGUF file, a missing binary, or startup flags the installed llama-server rejects.

Source

Thrown at studio/backend/core/rag/embed_llama_server.py:615

            **windows_hidden_subprocess_kwargs(),
            **child_popen_kwargs(),
        )
        self._process = proc
        # Long-lived, and child_popen_kwargs() is empty on macOS, so the crash
        # record is the only thing that can reap it after a force quit.
        adopt_pid(proc.pid)
        self._port = port
        self._stdout_thread = threading.Thread(
            target = self._drain_stdout,
            args = (proc,),
            daemon = True,
            name = "llama-embed-stdout",
        )
        self._stdout_thread.start()
        if not self._wait_for_health(config.EMBED_STARTUP_TIMEOUT_S):
            tail = "\n".join(self._stdout_lines[-30:])
            self._kill_process()
            raise RuntimeError(
                f"llama-server embedder failed to become healthy. Last output:\n{tail[:2000]}"
            )

    @staticmethod
    def _find_free_port() -> int:
        from core.inference.llama_cpp import LlamaCppBackend
        return LlamaCppBackend._find_free_port()

    def _wait_for_health(
        self,
        timeout: float,
        interval: float = 0.5,
    ) -> bool:
        """Poll /health until 200; bail early if the process exits."""
        deadline = time.monotonic() + timeout
        url = f"{self._base_url}/health"
        while time.monotonic() < deadline:
            if not self._process_alive():

View on GitHub (pinned to 203007d190)

Solutions

1. Read the 'Last output' tail in the message — it contains llama-server's own startup error (e.g. 'unknown argument', 'GGUF header mismatch', CUDA errors).
2. Verify the GGUF is an embedding model and the path in EMBED_MODEL_PATH exists: run llama-server manually with the same flags the backend uses.
3. Increase EMBED_STARTUP_TIMEOUT_S for large models on slow storage.
4. If torch/sentence-transformers works on this machine, set RAG_EMBED_BACKEND=sentence-transformers to bypass llama-server entirely.
5. Reinstall/upgrade the bundled llama-server binary so its CLI matches what the backend expects.

Example fix

# before
RAG_EMBED_STARTUP_TIMEOUT_S=15  # too short for a 7B GGUF on cold disk

# after
RAG_EMBED_STARTUP_TIMEOUT_S=120  # large embedding GGUFs can take minutes to mmap+warm

import shutil
from core import config

def llama_server_precheck() -> list[str]:
    problems = []
    model = getattr(config, "EMBED_MODEL_PATH", None)
    if not model or not Path(model).is_file():
        problems.append(f"EMBED_MODEL_PATH missing or not a file: {model!r}")
    if shutil.which("llama-server") is None and not getattr(config, "LLAMA_SERVER_BIN", ""):
        problems.append("llama-server binary not found on PATH")
    return problems  # empty list -> safe to construct LlamaServerBackend

try:
    backend = LlamaServerBackend()
except RuntimeError as e:
    if "failed to become healthy" not in str(e):
        raise
    log.error("embedder startup failed; server output: %s", e)
    backend = build_sentence_transformers_backend()  # fallback

When it happens

Trigger: Constructing LlamaServerBackend (or first encode/dim call that triggers _ensure_ready -> spawn) with config.EMBED_MODEL_PATH pointing at a bad GGUF, an incompatible llama-server binary, insufficient VRAM for the configured n_gpu_layers, or EMBED_STARTUP_TIMEOUT_S set too low for a large model to load.

Common situations: Switching RAG_EMBED_BACKEND to 'llama-server' with a GGUF that is not an embedding model (e.g. a chat model without an embedding head); upgrading llama.cpp which changed CLI flags; first-cold-start on a slow disk where model loading exceeds the timeout; CUDA driver mismatch causing the subprocess to abort at startup.

Related errors

• llama-server embedder POST {path} failed after retry
• Timeout waiting for '{expected_type}' response (no activity
• sd-server failed to become ready. Last output: {tail[:2000]}
• The local transcription runtime did not start in time.
• llama-server embedder POST {path} -> {e.response.status_code

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/b049d20b364090b2. Report an issue: GitHub.