{"record":{"id":"936b687433af8d15","repo":"unslothai/unsloth","slug":"llama-server-embedder-post-path-failed-after-ret","errorCode":null,"errorMessage":"llama-server embedder POST {path} failed after retry","messagePattern":"llama-server embedder POST (.+?) failed after retry","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/rag/embed_llama_server.py","lineNumber":728,"sourceCode":"        wedges a request); a fresh server unsticks both.\"\"\"\n        last_exc: Exception | None = None\n        for attempt in range(2):\n            self._ensure_ready()\n            try:\n                resp = self._client.post(f\"{self._base_url}{path}\", json = payload)\n                resp.raise_for_status()\n                return resp.json()\n            except (*_TRANSPORT_ERRORS, httpx.TimeoutException) as e:\n                last_exc = e\n                if attempt == 0:\n                    self._restart()\n                    continue\n            except httpx.HTTPStatusError as e:\n                body = e.response.text[:500] if e.response is not None else \"\"\n                raise RuntimeError(\n                    f\"llama-server embedder POST {path} -> {e.response.status_code}: {body}\"\n                ) from e\n        raise RuntimeError(f\"llama-server embedder POST {path} failed after retry\") from last_exc\n\n    def encode(\n        self,\n        texts,\n        *,\n        model_name = None,\n        normalize = True,\n    ):\n        \"\"\"Embed texts -> (N, dim) float32. ``model_name`` is ignored (the GGUF is\n        fixed by config). Normalizes in Python to match the ST backend.\"\"\"\n        n = len(texts)\n        if n == 0:\n            return np.zeros((0, self.dim()), dtype = np.float32)\n        rows: list[list[float]] = []\n        batch = max(1, config.EMBED_BATCH)\n        for start in range(0, n, batch):\n            chunk = list(texts[start : start + batch])\n            data = self._post(","sourceCodeStart":710,"sourceCodeEnd":746,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/rag/embed_llama_server.py#L710-L746","documentation":"Raised when a POST to the llama-server embedder fails with a transport-level error (connection refused/reset, timeout) on both attempts — the first failure triggers a full _restart() of the subprocess, and the second failure exhausts the retry loop. The original transport exception is chained as __cause__. It means the llama-server process is dying or unreachable even after a clean respawn.","triggerScenarios":"Calling encode()/dim() when the llama-server process crashed (OOM-killed, segfaulted on a bad GGUF); the health endpoint passed but the server dies on the first real inference; network/socket issues on the loopback connection; a restart race where the second POST is issued before the respawned server is ready.","commonSituations":"VRAM/RAM exhaustion killing llama-server mid-batch; a GGUF that loads but segfaults on specific inputs; concurrent encode() calls racing the restart path; OS-level resource limits (file descriptors) on long-running workers.","solutions":["Inspect e.__cause__ of the RuntimeError to see the underlying transport error (ConnectionReset vs ReadTimeout vs ConnectError).","Check whether llama-server is being OOM-killed (dmesg / Windows Event Log) and reduce model size, n_gpu_layers, or EMBED_BATCH.","Run llama-server manually and POST the same payload to reproduce the crash; if it segfaults, the GGUF or binary is bad.","Catch this error at the caller and fall back to the sentence-transformers backend via RAG_EMBED_BACKEND.","Reduce concurrency so multiple encode() calls do not race the single-instance restart."],"exampleFix":"# before\nvec = backend.encode([doc])  # unguarded; crash of llama-server surfaces raw\n\n# after\ntry:\n    vec = backend.encode([doc])\nexcept RuntimeError as e:\n    if \"failed after retry\" not in str(e):\n        raise\n    logger.warning(\"llama-server embedder unstable; falling back\")\n    vec = _st_fallback().encode([doc])","handlingStrategy":"retry","validationCode":"import httpx\n\ndef embedder_reachable(base_url: str, timeout: float = 2.0) -> bool:\n    try:\n        return httpx.get(f\"{base_url}/health\", timeout=timeout).status_code == 200\n    except (httpx.TransportError, httpx.TimeoutException):\n        return False","typeGuard":null,"tryCatchPattern":"for attempt in range(3):\n    try:\n        vecs = backend.encode(texts)\n        break\n    except RuntimeError as e:\n        if \"failed after retry\" not in str(e):\n            raise\n        if attempt == 2:\n            vecs = st_fallback_backend().encode(texts)\n        else:\n            backoff(2 ** attempt)","preventionTips":["Monitor the llama-server subprocess for OOM kills and cap --n-gpu-layers / batch size to fit VRAM.","Serialize encode() calls or rate-limit concurrency so restarts don't race new requests.","Wire the sentence-transformers backend as a configured fallback for embedding availability."],"tags":["network","retry","subprocess","embeddings","llama-cpp"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}