BerriAI/litellm · critical · ServiceUnavailableError

OllamaException: {original_exception}

Error message

OllamaException: {original_exception}

What it means

litellm maps Ollama 'Failed to establish a new connection' errors to litellm.ServiceUnavailableError. The HTTP client could not reach the Ollama server at the configured address - it is not running or is unreachable.

Source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:1816

    custom_llm_provider: str,
    error_str: str,
    exception_type: str,
    exception_provider: str,
    extra_information: str,
) -> None:
    if isinstance(original_exception, dict):
        error_str = original_exception.get("error", "")
    else:
        error_str = str(original_exception)
    if "no such file or directory" in error_str:
        raise BadRequestError(
            message=f"OllamaException: Invalid Model/Model not loaded - {original_exception}",
            model=model,
            llm_provider="ollama",
            response=getattr(original_exception, "response", None),
        )
    elif "Failed to establish a new connection" in error_str:
        raise ServiceUnavailableError(
            message=f"OllamaException: {original_exception}",
            llm_provider="ollama",
            model=model,
            response=getattr(original_exception, "response", None),
        )
    elif "Invalid response object from API" in error_str:
        raise BadRequestError(
            message=f"OllamaException: {original_exception}",
            llm_provider="ollama",
            model=model,
            response=getattr(original_exception, "response", None),
        )
    elif "Read timed out" in error_str:
        raise Timeout(
            message=f"OllamaException: {original_exception}",
            llm_provider="ollama",
            model=model,
        )

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Start the server: run `ollama serve` (or ensure the service/daemon is up)
2. Set OLLAMA_API_BASE (or api_base=) to the correct reachable URL, including scheme
3. If Ollama must accept remote clients, set OLLAMA_HOST=0.0.0.0 and open the port
4. From Docker, use host.docker.internal:11434 or the host network on Linux
5. Verify reachability: curl http://<host>:11434/api/tags

Example fix

# before
litellm.completion(model='ollama/llama3', messages=msgs)  # nothing on :11434
# after
import os
os.environ['OLLAMA_API_BASE'] = 'http://host.docker.internal:11434'
# shell: OLLAMA_HOST=0.0.0.0 ollama serve
litellm.completion(model='ollama/llama3', messages=msgs)

import requests, os
base = os.environ.get('OLLAMA_API_BASE', 'http://localhost:11434')
try:
    requests.get(f'{base}/api/tags', timeout=3).raise_for_status()
except requests.ConnectionError:
    raise RuntimeError(f'Ollama unreachable at {base} - start `ollama serve`')

import litellm

def is_ollama_down(e: Exception) -> bool:
    return isinstance(e, litellm.ServiceUnavailableError) and 'new connection' in str(e)

try:
    litellm.completion(model='ollama/llama3', messages=msgs)
except litellm.ServiceUnavailableError as e:
    if 'new connection' in str(e):
        wait_for_ollama_health()  # then retry once
    raise

When it happens

Trigger: Calling model='ollama/...' when `ollama serve` is not running, the host/port is wrong (default http://localhost:11434), a firewall blocks it, or the container has no route to the host (Docker host.docker.internal issues).

Common situations: Forgot to start the daemon; OLLAMA_API_BASE pointing at a stale IP; macOS/Windows Docker containers unable to reach localhost services; Ollama bound only to 127.0.0.1 while client is remote.

Related errors

• OllamaException: Invalid Model/Model not loaded - {original_
• VLLMException - {original_exception.message}
• Unclassified keys in {PRICES_PATH.name}: {', '.join(unclassi
• No Braintrust API token provided. Pass via Authorization hea
• Braintrust API error: {e.response.text}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/b1e02662b53942e6. Report an issue: GitHub.