BerriAI/litellm · error · ValueError

api_base not set for LiteLLM Proxy route. Set in env via `LI

Error message

api_base not set for LiteLLM Proxy route. Set in env via `LITELLM_PROXY_API_BASE`

What it means

When listing models from a downstream LiteLLM Proxy (to populate a client-side model list), the provider resolves api_base from the argument or LITELLM_PROXY_API_BASE. If both are unset, get_models raises ValueError because there is no default proxy address to query {api_base}/models.

Source

Thrown at litellm/llms/litellm_proxy/chat/transformation.py:48

        supported_openai_params: Final = self.get_supported_openai_params(model)
        for param, value in non_default_params.items():
            if param == "thinking":
                optional_params.setdefault("extra_body", {})["thinking"] = value
            elif param in supported_openai_params:
                optional_params[param] = value
        return optional_params

    def _get_openai_compatible_provider_info(
        self, api_base: str | None, api_key: str | None
    ) -> tuple[str | None, str | None]:
        api_base = api_base or get_secret_str("LITELLM_PROXY_API_BASE")
        dynamic_api_key: Final = api_key or get_secret_str("LITELLM_PROXY_API_KEY")
        return api_base, dynamic_api_key

    def get_models(self, api_key: str | None = None, api_base: str | None = None) -> list[str]:
        api_base, api_key = self._get_openai_compatible_provider_info(api_base, api_key)
        if api_base is None:
            raise ValueError("api_base not set for LiteLLM Proxy route. Set in env via `LITELLM_PROXY_API_BASE`")
        models: Final = super().get_models(api_key=api_key, api_base=api_base)
        return [f"litellm_proxy/{model}" for model in models]

    @staticmethod
    def get_api_key(api_key: str | None = None) -> str | None:
        return api_key or get_secret_str("LITELLM_PROXY_API_KEY")

    @staticmethod
    def _should_use_litellm_proxy_by_default(
        litellm_params: LiteLLM_Params | None = None,
    ):
        """
        Returns True if litellm proxy should be used by default for a given request

        Issue: https://github.com/BerriAI/litellm/issues/10559

        Use case:
        - When using Google ADK, users want a flag to dynamically enable sending the request to litellm proxy or not

View on GitHub (pinned to 6c2dcb801b)

Solutions

Set LITELLM_PROXY_API_BASE to the upstream proxy's base URL (e.g. http://proxy-b:4000)
Or pass api_base explicitly to get_models / the model entry's litellm_params
Also set LITELLM_PROXY_API_KEY for authentication against the upstream proxy

Example fix

# before
models = provider.get_models(api_key="sk-upstream")

# after
import os
os.environ["LITELLM_PROXY_API_BASE"] = "http://proxy-b:4000"
os.environ["LITELLM_PROXY_API_KEY"] = "sk-upstream"
models = provider.get_models()

Defensive patterns

Strategy: validation

Validate before calling

import os

def ensure_litellm_proxy_base() -> str:
    base = os.getenv("LITELLM_PROXY_API_BASE")
    if not base:
        raise ValueError("LITELLM_PROXY_API_BASE not configured; cannot list upstream proxy models")
    return base.rstrip("/")

Try / catch

try:
    models = provider.get_models()
except ValueError as e:
    if "api_base not set for LiteLLM Proxy" in str(e):
        raise RuntimeError("Proxy chaining misconfigured: missing upstream LITELLM_PROXY_API_BASE") from e

Prevention

In proxy-to-proxy setups, configure LITELLM_PROXY_API_BASE and LITELLM_PROXY_API_KEY as a pair
Smoke-test {base}/models with the upstream key after any networking change
Assert required env vars at process start rather than failing lazily on first model listing

When it happens

Trigger: Chaining LiteLLM instances (proxy-to-proxy) and calling get_models for the litellm_proxy provider without api_base or LITELLM_PROXY_API_BASE; env var not propagated to the pod querying the upstream proxy.

Common situations: Multi-hop setups (SDK → proxy A → proxy B) where proxy A lacks LITELLM_PROXY_API_BASE; variable misspelled; env set at build time but not runtime.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/c7dda0f1e2c96bed. Report an issue: GitHub.