{"record":{"id":"f11de53af0335abb","repo":"unslothai/unsloth","slug":"llama-server-embedder-post-path-e-response-s","errorCode":null,"errorMessage":"llama-server embedder POST {path} -> {e.response.status_code}: {body}","messagePattern":"llama-server embedder POST (.+?) -> (.+?): (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/rag/embed_llama_server.py","lineNumber":725,"sourceCode":"    def _post(self, path: str, payload: dict) -> dict:\n        \"\"\"POST to the server, restarting once and retrying on a dropped connection\n        (the reaper may have killed us) or a timeout (the bundled build sometimes\n        wedges a request); a fresh server unsticks both.\"\"\"\n        last_exc: Exception | None = None\n        for attempt in range(2):\n            self._ensure_ready()\n            try:\n                resp = self._client.post(f\"{self._base_url}{path}\", json = payload)\n                resp.raise_for_status()\n                return resp.json()\n            except (*_TRANSPORT_ERRORS, httpx.TimeoutException) as e:\n                last_exc = e\n                if attempt == 0:\n                    self._restart()\n                    continue\n            except httpx.HTTPStatusError as e:\n                body = e.response.text[:500] if e.response is not None else \"\"\n                raise RuntimeError(\n                    f\"llama-server embedder POST {path} -> {e.response.status_code}: {body}\"\n                ) from e\n        raise RuntimeError(f\"llama-server embedder POST {path} failed after retry\") from last_exc\n\n    def encode(\n        self,\n        texts,\n        *,\n        model_name = None,\n        normalize = True,\n    ):\n        \"\"\"Embed texts -> (N, dim) float32. ``model_name`` is ignored (the GGUF is\n        fixed by config). Normalizes in Python to match the ST backend.\"\"\"\n        n = len(texts)\n        if n == 0:\n            return np.zeros((0, self.dim()), dtype = np.float32)\n        rows: list[list[float]] = []\n        batch = max(1, config.EMBED_BATCH)","sourceCodeStart":707,"sourceCodeEnd":743,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/rag/embed_llama_server.py#L707-L743","documentation":"Raised when a POST to the llama-server embedder (e.g. /v1/embeddings) returns a non-2xx HTTP status. Unlike transport errors, HTTP status errors are not retried — the backend immediately converts the httpx.HTTPStatusError into a RuntimeError carrying the status code and up to 500 chars of the response body. It usually indicates a request the running server cannot serve, such as a context-length overflow or malformed JSON payload.","triggerScenarios":"Calling encode() with a batch whose combined prompt exceeds the server's context window (400/413); posting to /v1/embeddings after the server was restarted with a different model so the 'embedding' model name or input shape is rejected (404/400); server under memory pressure returning 500.","commonSituations":"EMBED_BATCH set larger than the server's --ctx-size divided by typical chunk length; a chunking change that produced very long texts; version skew between the client's request format and the llama-server build; server restarted mid-session with different flags.","solutions":["Read the status code and body snippet in the message — llama-server states the exact reason (e.g. 'prompt too long', 'model not found').","Lower config.EMBED_BATCH so one batch's token count fits within the server's context size.","Pre-truncate/normalize input texts before encode() so no single text exceeds the context budget.","Ensure the llama-server subprocess was started with a --ctx-size at least as large as batch_size * max_text_tokens.","Restart the backend (it re-spawns the server) if the server has drifted into a bad state."],"exampleFix":"# before\nbatch = max(1, config.EMBED_BATCH)  # e.g. 512 texts at once overflows ctx\n\n# after\nbatch = max(1, min(config.EMBED_BATCH, 64))  # keep batch token footprint well under ctx-size","handlingStrategy":"validation","validationCode":"from core import config\n\nMAX_TOKENS_PER_TEXT = 8192  # keep well under server ctx-size\n\ndef batch_fits_context(texts: list[str]) -> bool:\n    per_call = max(1, config.EMBED_BATCH)\n    worst = max((len(t) for t in texts), default=0)\n    return per_call * (worst // 3 + 1) < 8192  # ~3 chars/token heuristic","typeGuard":null,"tryCatchPattern":"try:\n    data = backend._post(\"/v1/embeddings\", payload)\nexcept RuntimeError as e:\n    if \"-> 4\" in str(e) or \"-> 5\" in str(e):  # 4xx/5xx surfaced\n        log.error(\"embedder rejected batch: %s\", e)\n        raise EmbeddingRequestError(str(e)) from e\n    raise","preventionTips":["Bound each text's length before encode() so a single input cannot exceed the context window.","Keep EMBED_BATCH small enough that batch_tokens << ctx-size.","Log the response body fragment from the message — it names the exact server-side reason."],"tags":["http","embeddings","llama-cpp","batching"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}