BerriAI/litellm · error · ValueError

api_base not set for Hosted VLLM responses API. Set via api_

Error message

api_base not set for Hosted VLLM responses API. Set via api_base parameter or HOSTED_VLLM_API_BASE environment variable

What it means

Raised by HostedVLLM ResponsesAPIConfig.get_complete_url when neither the api_base argument nor the HOSTED_VLLM_API_BASE environment variable is set. The /v1/responses API handler (distinct from the chat/embedding handlers) needs your self-hosted vLLM server URL and fails fast with a ValueError describing both resolution routes.

Source

Thrown at litellm/llms/hosted_vllm/responses/transformation.py:56

        api_key: Final = (
            litellm_params.api_key or get_secret_str("HOSTED_VLLM_API_KEY") or "fake-api-key"
        )  # vllm does not require an api key
        headers.update(
            {
                "Authorization": f"Bearer {api_key}",
            }
        )
        return headers

    def get_complete_url(
        self,
        api_base: str | None,
        litellm_params: dict,
    ) -> str:
        api_base = api_base or get_secret_str("HOSTED_VLLM_API_BASE")

        if api_base is None:
            raise ValueError(
                "api_base not set for Hosted VLLM responses API. "
                "Set via api_base parameter or HOSTED_VLLM_API_BASE environment variable"
            )

        # Remove trailing slashes
        api_base = api_base.rstrip("/")

        # If api_base already ends with /v1, append /responses
        # Otherwise append /v1/responses
        if api_base.endswith("/v1"):
            return f"{api_base}/responses"

        return f"{api_base}/v1/responses"

    def supports_native_websocket(self) -> bool:
        """Hosted vLLM does not support native WebSocket for Responses API"""
        return False

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Set export HOSTED_VLLM_API_BASE=http://vllm-host:8000 (the code appends /responses or /v1/responses depending on whether the base already ends with /v1).
2. Or pass api_base='http://vllm-host:8000' to the litellm.responses call.
3. Confirm your vLLM build actually serves the /v1/responses endpoint (recent vLLM versions only) before relying on this path.

Example fix

# before
litellm.responses(model='hosted_vllm/qwen3-32b', input='hello')
# raises ValueError: api_base not set for Hosted VLLM responses API

# after
litellm.responses(
    model='hosted_vllm/qwen3-32b',
    input='hello',
    api_base='http://localhost:8000',  # -> http://localhost:8000/v1/responses
)

import os

def vllm_responses_base(api_base: str | None = None) -> str:
    base = api_base or os.environ.get("HOSTED_VLLM_API_BASE")
    if not base:
        raise ValueError("api_base or HOSTED_VLLM_API_BASE required for hosted_vllm responses API")
    return base

When it happens

Trigger: Calling litellm.responses(model='hosted_vllm/...', ...) with no api_base and no HOSTED_VLLM_API_BASE env var. Note this handler DOES honor the env var (unlike the rerank handler) — the error only fires when both sources are empty.

Common situations: Env var set in the chat service container but the responses call runs elsewhere (job/worker without the var); experimenting with the new Responses API on an existing hosted_vllm setup where api_base was always passed per-call before; typo in the env var name.

Related errors

• api_base is required for hosted_vllm embeddings
• api_base is required for Azure WebSocket
• api_base is required
• No api base was set. Please provide an api_base, or set the
• api_base must be provided for Hosted VLLM rerank

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/291b3e75c959220f. Report an issue: GitHub.