BerriAI/litellm · error · ValueError

api_base not set for LiteLLM Proxy responses API. Set via…

Error message

api_base not set for LiteLLM Proxy responses API. Set via api_base parameter or LITELLM_PROXY_API_BASE environment variable

What it means

The Responses-API transformation for the litellm_proxy provider routes requests to the upstream proxy's {api_base}/responses endpoint. api_base is taken from the call parameter or the LITELLM_PROXY_API_BASE secret/env var; if neither is present it raises ValueError (note this variant also disables native WebSocket support).

Solutions

Set LITELLM_PROXY_API_BASE to the upstream proxy root (the code appends /responses)
Or pass api_base explicitly on the litellm.responses call / deployment entry
Ensure the upstream proxy version supports the /responses endpoint
Restart the calling process after setting the variable

Example fix

# before
litellm.responses(model="litellm_proxy/gpt-4o", input="hello")  # ValueError

# after
os.environ["LITELLM_PROXY_API_BASE"] = "http://proxy-b:4000"
os.environ["LITELLM_PROXY_API_KEY"] = "sk-..."
litellm.responses(model="litellm_proxy/gpt-4o", input="hello")

Defensive patterns

Strategy: validation

Validate before calling

import os

if not os.getenv("LITELLM_PROXY_API_BASE"):
    raise SystemExit("Set LITELLM_PROXY_API_BASE to use litellm_proxy responses API")

Try / catch

try:
    litellm.responses(model="litellm_proxy/gpt-4o", input="hi")
except ValueError as e:
    if "api_base not set" in str(e):
        raise ConfigError("Set LITELLM_PROXY_API_BASE for the responses API") from e
    raise

Prevention

When adopting the Responses API on chained proxies, reuse the same LITELLM_PROXY_API_BASE/KEY setup as chat
Confirm the upstream proxy version serves /responses before routing traffic
Validate chaining env vars at startup for every API surface you enable

When it happens

Trigger: Calling litellm.responses(model="litellm_proxy/gpt-4o", input=...) or a proxy route that forwards to a litellm_proxy responses deployment with no api_base configured and no LITELLM_PROXY_API_BASE set.

Common situations: Adopting the newer Responses API on a chained-proxy setup that only configured chat credentials; missing env var in the new service that uses responses.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/963c5ed8752c2643. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/litellm_proxy/responses/transformation.py:38

    @property
    def custom_llm_provider(self) -> LlmProviders:
        return LlmProviders.LITELLM_PROXY

    def get_complete_url(
        self,
        api_base: str | None,
        litellm_params: dict,
    ) -> str:
        """
        Get the endpoint for LiteLLM Proxy responses API.

        Uses LITELLM_PROXY_API_BASE environment variable if api_base is not provided.
        """
        api_base = api_base or get_secret_str("LITELLM_PROXY_API_BASE")

        if api_base is None:
            raise ValueError(
                "api_base not set for LiteLLM Proxy responses API. "
                "Set via api_base parameter or LITELLM_PROXY_API_BASE environment variable"
            )

        # Remove trailing slashes
        api_base = api_base.rstrip("/")

        return f"{api_base}/responses"

    def supports_native_websocket(self) -> bool:
        """LiteLLM Proxy does not support native WebSocket for Responses API"""
        return False

View on GitHub (pinned to 77b7c6c40c)