BerriAI/litellm · error · ValueError

VLLM_API_BASE is not set. Please set the environment variabl

Error message

VLLM_API_BASE is not set. Please set the environment variable, to use VLLM's pass-through - `{LITELLM_API_BASE}/vllm/{endpoint}`.

What it means

Raised by VLLMModelInfo.get_api_base when neither the api_base argument nor the VLLM_API_BASE environment variable is set. LiteLLM's vLLM pass-through routes ({LITELLM_API_BASE}/vllm/{endpoint}) must proxy to a running vLLM OpenAI-compatible server, and its address has to come from one of those two places.

Source

Thrown at litellm/llms/vllm/common_utils.py:50

    def validate_environment(
        self,
        headers: dict,
        model: str,
        messages: list[AllMessageValues],
        optional_params: dict,
        litellm_params: dict,
        api_key: str | None = None,
        api_base: str | None = None,
    ) -> dict:
        if api_key is not None:
            headers["x-api-key"] = api_key
        return headers

    @staticmethod
    def get_api_base(api_base: str | None = None) -> str | None:
        api_base = api_base or get_secret_str("VLLM_API_BASE")
        if api_base is None:
            raise ValueError(
                "VLLM_API_BASE is not set. Please set the environment variable, to use VLLM's pass-through - `{LITELLM_API_BASE}/vllm/{endpoint}`."
            )
        return api_base

    @staticmethod
    def get_api_key(api_key: str | None = None) -> str | None:
        return None

    @staticmethod
    def get_base_model(model: str) -> str | None:
        return model

    def get_models(self, api_key: str | None = None, api_base: str | None = None) -> list[str]:
        api_base = VLLMModelInfo.get_api_base(api_base)
        api_key = VLLMModelInfo.get_api_key(api_key)
        endpoint: Final = "/v1/models"
        if api_base is None or api_key is None:
            raise ValueError(

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Export VLLM_API_BASE pointing at your vLLM server, e.g. export VLLM_API_BASE=http://localhost:8000
2. Or pass api_base explicitly in the litellm client/request (litellm.client or route config)
3. In containers, add the variable to the environment section and restart the proxy
4. Verify with: python -c "import os; print(os.environ.get('VLLM_API_BASE'))"

Example fix

# before
# nothing set -> ValueError on /vllm/* calls

# after
export VLLM_API_BASE=http://localhost:8000

# or per call
client.chat.completions.create(
    model="vllm/meta-llama/Meta-Llama-3-8B-Instruct",
    messages=[...],
    base_url="http://localhost:8000",  # or api_base depending on client
    api_key="none",
)

import os

VLLM_URL = os.environ.get("VLLM_API_BASE") or "http://localhost:8000"

if not os.environ.get("VLLM_API_BASE"):
    raise ConfigError("Set VLLM_API_BASE before using the /vllm pass-through")

def has_vllm_api_base() -> bool:
    import os
    return bool(os.environ.get("VLLM_API_BASE"))

try:
    litellm.Router(model_list=[{"model_name": "vllm/llama3", "litellm_params": {"model": "vllm/llama3"}}])
except ValueError as e:
    if "VLLM_API_BASE is not set" in str(e):
        os.environ["VLLM_API_BASE"] = "http://localhost:8000"  # then retry once
    else:
        raise

When it happens

Trigger: Using the /vllm pass-through (e.g. /vllm/v1/chat/completions or /vllm/models) without exporting VLLM_API_BASE and without passing api_base in the request/client config.

Common situations: Fresh deployments missing the env var in docker-compose/k8s; CI runs that never set VLLM_API_BASE; env vars set in a different shell or dropped by systemd/supervisor; typo'd variable name (VLLM_API_URL).

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors

• custom_ui_sso_sign_in_handler is not configured. Please set
• Invalid mode: {custom_auth_settings['mode']}
• LLM Router not initialized. Ensure models added to proxy.
• DB not connected. This endpoint needs a database; set DATABA
• S3 bucket name is required. Set 's3_bucket_name' parameter o

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/7869a04dfd908c29. Report an issue: GitHub.