What causes "VLLM api base not found" in BerriAI/litellm?

Set VLLM_API_BASE in the environment, e.g. export VLLM_API_BASE=http://localhost:8000.

BerriAI/litellm · error · Exception

VLLM api base not found

Error message

VLLM api base not found

What it means

Raised by the vLLM passthrough config when building the target URL for a passthrough request (e.g. client.vllm endpoints or /vllm/{endpoint} routes on the LiteLLM proxy). get_api_base() resolves api_base from the argument or the VLLM_API_BASE environment variable; if both are unset the URL cannot be constructed and a plain Exception('VLLM api base not found') is raised. Note: the underlying get_api_base in litellm/llms/vllm/common_utils.py:47-53 usually raises its own ValueError mentioning VLLM_API_BASE, so this line is the defensive backstop.

Source

Thrown at litellm/llms/vllm/passthrough/transformation.py:27


class VLLMPassthroughConfig(VLLMModelInfo, BasePassthroughConfig):
    def is_streaming_request(self, endpoint: str, request_data: dict) -> bool:
        return "stream" in request_data

    def get_complete_url(
        self,
        api_base: str | None,
        api_key: str | None,
        model: str,
        endpoint: str,
        request_query_params: dict | None,
        litellm_params: dict,
    ) -> tuple["URL", str]:
        base_target_url: Final = self.get_api_base(api_base)

        if base_target_url is None:
            raise Exception("VLLM api base not found")

        return (
            self.format_url(endpoint, base_target_url, request_query_params),
            base_target_url,
        )

View on GitHub (pinned to 77b7c6c40c)

Solutions

Set VLLM_API_BASE in the environment, e.g. export VLLM_API_BASE=http://localhost:8000.
Or pass api_base explicitly to the passthrough call / litellm_params.
On the LiteLLM proxy, add VLLM_API_BASE to the proxy's environment (environment_variables in config.yaml or the container env).
Verify with: echo $VLLM_API_BASE and curl $VLLM_API_BASE/v1/models to confirm the server is reachable.

Example fix

# before
resp = client.vllm.pooling(inputs=[[1.0, 2.0]])  # raises: VLLM api base not found

# after
import os
os.environ["VLLM_API_BASE"] = "http://localhost:8000"
resp = client.vllm.pooling(inputs=[[1.0, 2.0]])

Defensive patterns

Strategy: validation

Validate before calling

import os

VLLM_URL = os.getenv("VLLM_API_BASE") or os.getenv("api_base")
if not VLLM_URL:
    raise RuntimeError("VLLM_API_BASE is not set - required for vLLM passthrough calls")
# then safe to use passthrough endpoints
resp = client.vllm.pooling(inputs=[[1.0, 2.0]])

Type guard

const hasVllmBase = (cfg: Record<string, string | undefined>): boolean =>
  Boolean(cfg.VLLM_API_BASE ?? cfg.api_base);

Try / catch

try:
    resp = client.vllm.some_endpoint(...)
except Exception as e:
    if "api base not found" in str(e).lower():
        raise RuntimeError("Set VLLM_API_BASE before using the vLLM passthrough API") from e
    raise

Prevention

Set VLLM_API_BASE in the same deploy step that starts the vLLM server.
Fail fast at boot: assert os.environ.get('VLLM_API_BASE') in your app's config check.
Add a healthcheck that curls $VLLM_API_BASE/v1/models before accepting traffic.

When it happens

Trigger: Calling litellm.vllm passthrough methods (client.vllm.<endpoint>) without api_base and without VLLM_API_BASE set; proxy deployments using the /vllm passthrough route where the VLLM_API_BASE env var was never configured; passing api_base=None explicitly.

Common situations: Self-hosting a vLLM server and using the passthrough API while forgetting the env var; proxy config.yaml defines the model but the passthrough route needs the dedicated VLLM_API_BASE environment variable; port-forward changes leaving a stale/empty var.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/e1d99e6ab3b952a2. Report an issue: GitHub.