{"record":{"id":"50ff9bd14330680a","repo":"BerriAI/litellm","slug":"vllm-api-base-or-vllm-api-key-is-not-set-please-s","errorCode":null,"errorMessage":"VLLM_API_BASE or VLLM_API_KEY is not set. Please set the environment variable, to query VLLM's `/models` endpoint.","messagePattern":"VLLM_API_BASE or VLLM_API_KEY is not set\\. Please set the environment variable, to query VLLM's `/models` endpoint\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/vllm/common_utils.py","lineNumber":68,"sourceCode":"            raise ValueError(\n                \"VLLM_API_BASE is not set. Please set the environment variable, to use VLLM's pass-through - `{LITELLM_API_BASE}/vllm/{endpoint}`.\"\n            )\n        return api_base\n\n    @staticmethod\n    def get_api_key(api_key: str | None = None) -> str | None:\n        return None\n\n    @staticmethod\n    def get_base_model(model: str) -> str | None:\n        return model\n\n    def get_models(self, api_key: str | None = None, api_base: str | None = None) -> list[str]:\n        api_base = VLLMModelInfo.get_api_base(api_base)\n        api_key = VLLMModelInfo.get_api_key(api_key)\n        endpoint: Final = \"/v1/models\"\n        if api_base is None or api_key is None:\n            raise ValueError(\n                \"VLLM_API_BASE or VLLM_API_KEY is not set. Please set the environment variable, to query VLLM's `/models` endpoint.\"\n            )\n\n        url: Final = _add_path_to_api_base(api_base, endpoint)\n        response: Final = litellm.module_level_client.get(\n            url=url,\n        )\n\n        response.raise_for_status()\n\n        models: Final = response.json()[\"data\"]\n\n        return [model[\"id\"] for model in models]\n\n    def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BaseLLMException:\n        return VLLMError(status_code=status_code, message=error_message, headers=headers)\n","sourceCodeStart":50,"sourceCodeEnd":85,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/vllm/common_utils.py#L50-L85","documentation":"Raised by VLLMModelInfo.get_models when api_base or api_key is None after resolution. Critical detail: on this code path get_api_key() always returns None (litellm/llms/vllm/common_utils.py:56), so the check can never pass — get_models() for vLLM raises this ValueError unconditionally, even when VLLM_API_BASE and VLLM_API_KEY are both set. It is a library defect, not purely a configuration problem.","triggerScenarios":"Any call that lists models for the vllm provider — litellm.models() routed to vLLM, proxy model-listing for a vllm deployment, or get_models(api_base=...) — regardless of which environment variables are set.","commonSituations":"Proxy startup fetching model lists for a vllm provider; dashboards enumerating available models; users setting VLLM_API_KEY and still hitting the error because the getter ignores it.","solutions":["Do not rely on model listing for vllm; hardcode/declare the deployed model ids in your proxy config","Query the server directly: curl $VLLM_API_BASE/v1/models (vLLM needs no real API key)","Patch or subclass VLLMModelInfo.get_api_key to return api_key (or 'EMPTY') and pass the class through your integration","Check for a newer litellm release where the vllm get_models path is fixed before working around it"],"exampleFix":"# before\nmodels = provider_config.get_models(api_base=\"http://localhost:8000\")\n# always raises: VLLM_API_BASE or VLLM_API_KEY is not set\n\n# after (query directly; vLLM serves an OpenAI-compatible list)\nimport httpx\nresp = httpx.get(\"http://localhost:8000/v1/models\")\nresp.raise_for_status()\nmodels = [m[\"id\"] for m in resp.json()[\"data\"]]\n\n# or monkeypatch the broken getter\n# litellm.llms.vllm.common_utils.VLLMModelInfo.get_api_key = staticmethod(lambda api_key=None: api_key or \"EMPTY\")","handlingStrategy":"fallback","validationCode":"import os, httpx\n\ndef list_vllm_models_direct(api_base: str | None = None) -> list[str]:\n    base = api_base or os.environ.get(\"VLLM_API_BASE\")\n    if not base:\n        raise ConfigError(\"VLLM_API_BASE is not set\")\n    resp = httpx.get(f\"{base.rstrip('/')}/v1/models\", timeout=10)\n    resp.raise_for_status()\n    return [m[\"id\"] for m in resp.json()[\"data\"]]","typeGuard":null,"tryCatchPattern":"try:\n    models = provider_config.get_models(api_base=base)\nexcept ValueError as e:\n    if \"VLLM_API_BASE or VLLM_API_KEY\" in str(e):\n        # known litellm defect: get_api_key() always returns None -> always raises\n        models = [m[\"id\"] for m in httpx.get(f\"{base}/v1/models\").json()[\"data\"]]\n    else:\n        raise","preventionTips":["Declare vLLM model ids statically in proxy config instead of discovering them via get_models","Query $VLLM_API_BASE/v1/models directly — vLLM requires no real API key","Pin/track the litellm version and retest model listing after upgrades; check release notes for the vllm get_models fix"],"tags":["vllm","litellm","model-listing","library-bug","valueerror"],"backgroundTag":"missing-env-var","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}