BerriAI/litellm · error · ValueError

Failed to fetch models from Lemonade. Set Lemonade API Base…

Error message

Failed to fetch models from Lemonade. Set Lemonade API Base via `LEMONADE_API_BASE` environment variable. Error: {e}

What it means

Lemonade get_models performs a plain HTTP GET to {api_base}/models inside a try/except; any transport-level exception (connection refused, DNS resolution failure, timeout, SSL error) is re-raised as ValueError with the underlying error text appended. The message tells you the request never completed, not that the server rejected it.

Solutions

Confirm the server is up: curl http://$LEMONADE_API_BASE/models
Fix LEMONADE_API_BASE to the real host/port the server listens on
From Docker, use host.docker.internal (or the service name) instead of localhost
Match the scheme (http vs https) to what the server actually serves
Start the Lemonade/llama server if it is simply down

Example fix

# before: server not running on that host yet
os.environ["LEMONADE_API_BASE"] = "http://localhost:8000"
LemonadeConfig().get_models()  # ValueError: Failed to fetch models ... Error: [Errno 111] Connection refused

# after: verify reachability first, then list
import httpx
base = os.environ["LEMONADE_API_BASE"]
httpx.get(f"{base}/models", timeout=2).raise_for_status()  # fail fast with a clear cause
models = LemonadeConfig().get_models()

Defensive patterns

Strategy: retry

Validate before calling

import httpx, os

base = os.getenv("LEMONADE_API_BASE")
try:
    httpx.get(f"{base.rstrip('/')}/models", timeout=2).raise_for_status()
except httpx.HTTPError as e:
    raise RuntimeError(f"Lemonade server unreachable at {base}: {e}") from e

Try / catch

for attempt in range(3):
    try:
        models = LemonadeConfig().get_models()
        break
    except ValueError as e:
        if "Failed to fetch models" in str(e) and attempt < 2:
            time.sleep(1.5 * (attempt + 1))  # transient network blip: retry
            continue
        raise

Prevention

Start/health-check the Lemonade (llama.cpp) server before model discovery in scripts
Use the correct host from containers (host.docker.internal, not localhost)
Set explicit timeouts on discovery calls so hangs surface as fast failures

When it happens

Trigger: The Lemonade server is not running at LEMONADE_API_BASE; wrong host/port; firewall or VPN blocking the connection; HTTPS scheme against an HTTP-only local server; DNS name not resolvable.

Common situations: Local llama.cpp/Lemonade server not started before model discovery; port mismatch between the env var and the actual listener; container trying to reach localhost instead of host.docker.internal.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/318abedf7c0f5451. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/lemonade/chat/transformation.py:95

        Returns:
            List of model names prefixed with "lemonade/"
        """
        api_base, api_key = self._get_openai_compatible_provider_info(api_base=api_base, api_key=api_key)

        if api_base is None:
            raise ValueError(
                "LEMONADE_API_BASE is not set. Please set the environment variable to query Lemonade's /models endpoint."
            )

        # Getting the list of models from lemonade
        try:
            response: Final = litellm.module_level_client.get(
                url=f"{api_base}/models",
                headers=self._get_auth_headers(api_key),
            )
        except Exception as e:
            raise ValueError(
                f"Failed to fetch models from Lemonade. Set Lemonade API Base via `LEMONADE_API_BASE` environment variable. Error: {e}"
            )

        if response.status_code != 200:
            raise ValueError(
                f"Failed to fetch models from Lemonade. Status code: {response.status_code}, Response: {response.text}"
            )

        model_list: Final = response.json().get("data", [])
        return ["lemonade/" + model["id"] for model in model_list]

    @staticmethod
    def _get_positive_int(value: Any) -> int | None:
        if isinstance(value, bool):
            return None
        if isinstance(value, int) and value > 0:
            return value
        if isinstance(value, str):

View on GitHub (pinned to 77b7c6c40c)