BerriAI/litellm · error · ValueError

Failed to fetch models from Gemini. Status code: {response.s

Error message

Failed to fetch models from Gemini. Status code: {response.status_code}, Response: {response.json()}

What it means

After successfully authenticating, GeminiModelInfo.get_models() GETs {api_base}/{api_version}/models with the x-goog-api-key header. Any non-200 response (401/403 invalid key, 429 quota, 5xx outage, or a wrong GEMINI_API_BASE) raises this ValueError embedding the status code and Google's JSON error body.

Source

Thrown at litellm/llms/gemini/common_utils.py:397

            litellm_model_names.append(litellm_model_name)
        return litellm_model_names

    def get_models(self, api_key: str | None = None, api_base: str | None = None) -> list[str]:
        api_base = GeminiModelInfo.get_api_base(api_base)
        api_key = GeminiModelInfo.get_api_key(api_key)
        endpoint: Final = f"/{self.api_version}/models"
        if api_base is None or api_key is None:
            raise ValueError(
                "GEMINI_API_BASE or GEMINI_API_KEY/GOOGLE_API_KEY is not set. Please set the environment variable, to query Gemini's `/models` endpoint."
            )

        response: Final = litellm.module_level_client.get(
            url=f"{api_base}{endpoint}",
            headers={"x-goog-api-key": api_key},
        )

        if response.status_code != 200:
            raise ValueError(
                f"Failed to fetch models from Gemini. Status code: {response.status_code}, Response: {response.json()}"
            )

        models: Final = response.json()["models"]

        litellm_model_names: Final = self.process_model_name(models)
        return litellm_model_names

    def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BaseLLMException:
        return GeminiError(status_code=status_code, message=error_message, headers=headers)

    def get_token_counter(self) -> BaseTokenCounter | None:
        """
        Factory method to create a token counter for this provider.

        Returns:
            Optional TokenCounterInterface implementation for this provider,
            or None if token counting is not supported.

View on GitHub (pinned to 6c2dcb801b)

Solutions

Parse the status/body in the message: 401/403 -> fix the key; 429 -> back off or raise quota; 5xx -> retry later.
curl-verify the pair: curl -H "x-goog-api-key: $GOOGLE_API_KEY" https://generativelanguage.googleapis.com/v1beta/models.
Unset or correct GEMINI_API_BASE if it points anywhere other than Google's endpoint.

Example fix

# before
GEMINI_API_BASE="https://wrong-host.example.com"
models = litellm.get_model_list("gemini")

# after
GEMINI_API_BASE="https://generativelanguage.googleapis.com"
models = litellm.get_model_list("gemini")

Defensive patterns

Strategy: retry

Try / catch

import re, time

def list_gemini_models():
    for attempt in range(4):
        try:
            return litellm.get_model_list("gemini")
        except ValueError as e:
            if re.search(r"Status code: (429|5\\d\\d)", str(e)):
                time.sleep(2 ** attempt)
                continue
            raise
    raise RuntimeError("Gemini /models unavailable")

Prevention

Cache the discovered model list; don't re-query /models per request.
Verify GEMINI_API_BASE points at Google's host unless you intentionally proxy.
Monitor for 401 after key rotation — stale keys fail here first.

When it happens

Trigger: Expired/revoked AI Studio key; GEMINI_API_BASE pointing at a wrong host or an API version the key can't use; quota exhausted; transient Google 5xx during model discovery.

Common situations: Model-refresh cron jobs failing overnight after quota changes; rotated key not yet updated everywhere; self-hosted proxy used as GEMINI_API_BASE that returns an unexpected status.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/9c1dafde919d9914. Report an issue: GitHub.