BerriAI/litellm · error · ValueError

api_base not set for Hosted VLLM responses API. Set via…

Error message

api_base not set for Hosted VLLM responses API. Set via api_base parameter or HOSTED_VLLM_API_BASE environment variable

What it means

The hosted_vllm Responses API transformation requires an explicit base URL: get_complete_url() falls back to the HOSTED_VLLM_API_BASE environment variable and raises ValueError if both are unset. The URL is normalized (trailing slashes stripped; /responses appended to a base already ending in /v1, otherwise /v1/responses). As with all hosted_vllm routes, there is no default host because the server is user-deployed.

Solutions

1. Pass api_base: litellm.responses(model='hosted_vllm/<model>', api_base='http://localhost:8000/v1', input=...)
2. Or export HOSTED_VLLM_API_BASE=http://localhost:8000 (add /v1 if your base includes it — the suffix logic adapts)
3. Confirm the vLLM server was built with the Responses API enabled and serves /v1/responses (curl it)
4. Prefer a base ending in /v1 so the /responses suffix is appended consistently

Example fix

# before
litellm.responses(model="hosted_vllm/Qwen/Qwen3-32B", input="hello")

# after
litellm.responses(
    model="hosted_vllm/Qwen/Qwen3-32B",
    api_base="http://localhost:8000/v1",
    input="hello",
)

import os

base = api_base or os.getenv("HOSTED_VLLM_API_BASE")
if not base:
    raise SystemExit("hosted_vllm responses API needs api_base or HOSTED_VLLM_API_BASE")

When it happens

Trigger: litellm.responses(model='hosted_vllm/<model>', input=...) called without api_base and without HOSTED_VLLM_API_BASE in the environment.

Common situations: Trying LiteLLM's Responses API against a self-hosted vLLM instance and forgetting the base URL; env var present in dev but missing in CI/containers; pointing at a vLLM build that does not serve /v1/responses so other errors follow once the base is set.

Related errors

• api_base is required for hosted_vllm embeddings
• api_base must be provided for Hosted VLLM rerank
• api_base is required
• api_base must be provided for Hosted VLLM rerank
• api base needs to be a string. api_base=

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/291b3e75c959220f. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/hosted_vllm/responses/transformation.py:56

        api_key: Final = (
            litellm_params.api_key or get_secret_str("HOSTED_VLLM_API_KEY") or "fake-api-key"
        )  # vllm does not require an api key
        headers.update(
            {
                "Authorization": f"Bearer {api_key}",
            }
        )
        return headers

    def get_complete_url(
        self,
        api_base: str | None,
        litellm_params: dict,
    ) -> str:
        api_base = api_base or get_secret_str("HOSTED_VLLM_API_BASE")

        if api_base is None:
            raise ValueError(
                "api_base not set for Hosted VLLM responses API. "
                "Set via api_base parameter or HOSTED_VLLM_API_BASE environment variable"
            )

        # Remove trailing slashes
        api_base = api_base.rstrip("/")

        # If api_base already ends with /v1, append /responses
        # Otherwise append /v1/responses
        if api_base.endswith("/v1"):
            return f"{api_base}/responses"

        return f"{api_base}/v1/responses"

    def supports_native_websocket(self) -> bool:
        """Hosted vLLM does not support native WebSocket for Responses API"""
        return False

View on GitHub (pinned to 77b7c6c40c)