BerriAI/litellm · error · ValueError

api_base not set for LiteLLM Proxy responses API. Set via ap

Error message

api_base not set for LiteLLM Proxy responses API. Set via api_base parameter or LITELLM_PROXY_API_BASE environment variable

What it means

The Responses API transformation for the litellm_proxy provider forwards requests to {api_base}/responses on a downstream LiteLLM Proxy. If api_base is neither passed nor available via LITELLM_PROXY_API_BASE, it raises ValueError with instructions covering both resolution routes. Note this provider reports no native WebSocket support; HTTP(S) with a base URL is mandatory.

Source

Thrown at litellm/llms/litellm_proxy/responses/transformation.py:38

    @property
    def custom_llm_provider(self) -> LlmProviders:
        return LlmProviders.LITELLM_PROXY

    def get_complete_url(
        self,
        api_base: str | None,
        litellm_params: dict,
    ) -> str:
        """
        Get the endpoint for LiteLLM Proxy responses API.

        Uses LITELLM_PROXY_API_BASE environment variable if api_base is not provided.
        """
        api_base = api_base or get_secret_str("LITELLM_PROXY_API_BASE")

        if api_base is None:
            raise ValueError(
                "api_base not set for LiteLLM Proxy responses API. "
                "Set via api_base parameter or LITELLM_PROXY_API_BASE environment variable"
            )

        # Remove trailing slashes
        api_base = api_base.rstrip("/")

        return f"{api_base}/responses"

    def supports_native_websocket(self) -> bool:
        """LiteLLM Proxy does not support native WebSocket for Responses API"""
        return False

View on GitHub (pinned to 6c2dcb801b)

Solutions

Set api_base on the call or LITELLM_PROXY_API_BASE in the environment to the downstream proxy URL
For the LiteLLM proxy, put api_base in the model deployment's litellm_params so every responses call carries it
Restart the process after setting the env var and confirm it is visible to the process

Example fix

# before
resp = litellm.responses(model="litellm_proxy/gpt-4o", input="hi")

# after
resp = litellm.responses(
    model="litellm_proxy/gpt-4o",
    input="hi",
    api_base="http://downstream-proxy:4000",
    api_key="sk-upstream",
)

Defensive patterns

Strategy: validation

Validate before calling

import os

def ensure_responses_base(api_base: str | None = None) -> str:
    base = (api_base or os.getenv("LITELLM_PROXY_API_BASE") or "").rstrip("/")
    if not base:
        raise ValueError("Downstream proxy base required for the Responses API")
    return base

Try / catch

try:
    resp = litellm.responses(model="litellm_proxy/gpt-4o", input="hi", api_base=ensure_responses_base())
except ValueError as e:
    if "api_base not set for LiteLLM Proxy responses API" in str(e):
        raise RuntimeError("Responses route misconfigured: set api_base or LITELLM_PROXY_API_BASE") from e

Prevention

Remember this provider never uses native WebSockets — always provision an HTTP base URL
For the LiteLLM proxy, set api_base in the model deployment so SDK callers need nothing extra
Include a responses-api smoke test in deployment checks alongside chat completion tests

When it happens

Trigger: Calling litellm.responses(model="litellm_proxy/...", ...) without api_base while LITELLM_PROXY_API_BASE is unset — e.g. pointing a client's Responses API at another LiteLLM proxy that hasn't been configured.

Common situations: Proxy-to-proxy Responses API routing where only the key was configured; env var scoped to the wrong deployment; local .env not loaded in the server process.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/963c5ed8752c2643. Report an issue: GitHub.