BerriAI/litellm · error · ValueError

api_base not set for LiteLLM Proxy route. Set in env via…

Error message

api_base not set for LiteLLM Proxy route. Set in env via `LITELLM_PROXY_API_BASE`

What it means

The litellm_proxy provider forwards OpenAI-compatible calls to another LiteLLM proxy instance. Its get_models must query that upstream proxy, which requires knowing where it is: api_base argument or the LITELLM_PROXY_API_BASE secret/env var. When both are absent it raises ValueError before any HTTP call.

Solutions

Set LITELLM_PROXY_API_BASE to the upstream proxy URL (e.g. http://proxy-b:4000) in the calling process
Or add api_base directly to the litellm_proxy model deployment in the router/proxy config
Also set LITELLM_PROXY_API_KEY if the upstream proxy requires auth
Restart the process after setting the variable

Example fix

# before
from litellm.llms.litellm_proxy.chat.transformation import LiteLLMProxyChatConfig
LiteLLMProxyChatConfig().get_models()  # ValueError: api_base not set for LiteLLM Proxy route

# after
os.environ["LITELLM_PROXY_API_BASE"] = "http://proxy-b:4000"
os.environ["LITELLM_PROXY_API_KEY"] = "sk-..."
LiteLLMProxyChatConfig().get_models()

Defensive patterns

Strategy: validation

Validate before calling

import os

PROXY_BASE = os.getenv("LITELLM_PROXY_API_BASE")
if not PROXY_BASE:
    raise SystemExit("Set LITELLM_PROXY_API_BASE (URL of the upstream LiteLLM proxy) for litellm_proxy models")

Try / catch

try:
    models = LiteLLMProxyChatConfig().get_models()
except ValueError as e:
    if "api_base not set" in str(e):
        raise ConfigError("LITELLM_PROXY_API_BASE missing for proxy chaining") from e
    raise

Prevention

Configure api_base (and api_key) on every litellm_proxy deployment entry in router/proxy config
Treat LITELLM_PROXY_API_BASE/KEY as required secrets whenever chaining proxies
Validate chaining env vars at proxy startup

When it happens

Trigger: Model discovery on a litellm_proxy model (litellm.get_models / router model list, model names prefixed litellm_proxy/) with no api_base on the deployment and no LITELLM_PROXY_API_BASE set — the handler cannot know which proxy to ask for /models.

Common situations: Chaining proxies (proxy A routes to proxy B) but the env var was only configured on one; adding a litellm_proxy/ model to the router without an api_base field; fresh environment missing the chaining secrets.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/c7dda0f1e2c96bed. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/litellm_proxy/chat/transformation.py:48

        supported_openai_params: Final = self.get_supported_openai_params(model)
        for param, value in non_default_params.items():
            if param == "thinking":
                optional_params.setdefault("extra_body", {})["thinking"] = value
            elif param in supported_openai_params:
                optional_params[param] = value
        return optional_params

    def _get_openai_compatible_provider_info(
        self, api_base: str | None, api_key: str | None
    ) -> tuple[str | None, str | None]:
        api_base = api_base or get_secret_str("LITELLM_PROXY_API_BASE")
        dynamic_api_key: Final = api_key or get_secret_str("LITELLM_PROXY_API_KEY")
        return api_base, dynamic_api_key

    def get_models(self, api_key: str | None = None, api_base: str | None = None) -> list[str]:
        api_base, api_key = self._get_openai_compatible_provider_info(api_base, api_key)
        if api_base is None:
            raise ValueError("api_base not set for LiteLLM Proxy route. Set in env via `LITELLM_PROXY_API_BASE`")
        models: Final = super().get_models(api_key=api_key, api_base=api_base)
        return [f"litellm_proxy/{model}" for model in models]

    @staticmethod
    def get_api_key(api_key: str | None = None) -> str | None:
        return api_key or get_secret_str("LITELLM_PROXY_API_KEY")

    @staticmethod
    def _should_use_litellm_proxy_by_default(
        litellm_params: LiteLLM_Params | None = None,
    ):
        """
        Returns True if litellm proxy should be used by default for a given request

        Issue: https://github.com/BerriAI/litellm/issues/10559

        Use case:
        - When using Google ADK, users want a flag to dynamically enable sending the request to litellm proxy or not

View on GitHub (pinned to 77b7c6c40c)