{"record":{"id":"ff093be3c871639d","repo":"headroomlabs-ai/headroom","slug":"ollama-api-failed-after-self-max-retries-retrie","errorCode":null,"errorMessage":"Ollama API failed after {self._max_retries} retries: {last_error}","messagePattern":"Ollama API failed after (.+?) retries: (.+?)","errorType":"exception","errorClass":"ConnectionError","httpStatus":null,"severity":"error","filePath":"headroom/memory/adapters/embedders.py","lineNumber":935,"sourceCode":"                    self._detected_dimension = len(embedding)\n\n                return embedding\n\n            except (httpx.ConnectError, httpx.TimeoutException, httpx.HTTPStatusError) as e:\n                last_error = e\n                delay = self.RETRY_DELAY_BASE * (2**attempt)\n                logger.warning(\n                    f\"Ollama API error (attempt {attempt + 1}/{self._max_retries}): {e}. \"\n                    f\"Retrying in {delay:.1f}s...\"\n                )\n                await asyncio.sleep(delay)\n\n            except Exception as e:\n                # Non-retryable error\n                raise ConnectionError(f\"Ollama API error: {e}\") from e\n\n        # All retries exhausted\n        raise ConnectionError(\n            f\"Ollama API failed after {self._max_retries} retries: {last_error}\"\n        ) from last_error\n\n    async def embed(self, text: str) -> np.ndarray:\n        \"\"\"Generate an embedding for a single text.\n\n        Args:\n            text: The text to embed.\n\n        Returns:\n            Normalized embedding vector as float32 numpy array.\n\n        Raises:\n            ConnectionError: If API call fails after retries.\n        \"\"\"\n        # Handle empty string\n        if not text or not text.strip():\n            return np.zeros(self.dimension, dtype=np.float32)","sourceCodeStart":917,"sourceCodeEnd":953,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/memory/adapters/embedders.py#L917-L953","documentation":"Raised after OllamaEmbedder exhausts self._max_retries (default 3) attempts. Only httpx.ConnectError, TimeoutException, and HTTPStatusError are retried with backoff (0.5s base); this ConnectionError reports last_error and means the Ollama server stayed unreachable or slow across every attempt.","triggerScenarios":"Calling embed()/embed_batch() while the Ollama daemon is down (`ollama serve` not running), listening on a different host/port than base_url, blocked by firewall, or so overloaded (model loading, GPU contention) that every request times out.","commonSituations":"Ollama not started / crashed from OOM; base_url pointing at a remote machine that's off; first-call cold start loading a large model exceeding REQUEST_TIMEOUT each time; docker networking so localhost:11434 inside a container doesn't reach the host daemon.","solutions":["Confirm the daemon: curl http://localhost:11434/api/tags from the same host/network namespace as your app.","Start it: `ollama serve` (or restart the docker container with correct port mapping / host.docker.internal).","Pre-warm the model once (`curl /api/embed` or a tiny embed call) so later calls don't pay the model-load timeout.","Increase tolerance: OllamaEmbedder(max_retries=5) or a larger timeout if cold starts are slow.","Fix base_url to the address the daemon actually binds (it defaults to localhost)."],"exampleFix":"# before\nemb = OllamaEmbedder()  # ConnectionError: failed after 3 retries — server not running\n\n# after (shell)\nollama serve &\ncurl -s http://localhost:11434/api/tags  # verify, then rerun app","handlingStrategy":"retry","validationCode":"import httpx, os\n\ndef ollama_up(base_url: str = \"http://localhost:11434\") -> bool:\n    try:\n        return httpx.get(f\"{base_url}/api/tags\", timeout=3).status_code == 200\n    except httpx.HTTPError:\n        return False\n\nif not ollama_up(os.environ.get(\"OLLAMA_BASE_URL\", \"http://localhost:11434\")):\n    raise SystemExit(\"Ollama daemon unreachable; start it with `ollama serve`\")","typeGuard":null,"tryCatchPattern":"async def embed_retry(emb, text, tries=3):\n    for i in range(tries):\n        try:\n            return await emb.embed(text)\n        except ConnectionError as e:\n            if \"failed after\" in str(e) and i + 1 < tries:\n                await asyncio.sleep(10 * (i + 1))\n                continue\n            raise","preventionTips":["Health-check /api/tags at startup before accepting embedding work.","Use OllamaEmbedder(max_retries=5) plus KEEP_ALIVE settings to survive cold model loads.","In containers, point base_url at host.docker.internal (or the service name), not localhost."],"tags":["ollama","retry","network","embeddings","local-model"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}