BerriAI/litellm · warning · RateLimitError

litellm.RateLimitError: {custom_llm_provider}Exception - {er

Error message

litellm.RateLimitError: {custom_llm_provider}Exception - {error_str}

What it means

The Vertex error string matched a quota condition — '429 Quota exceeded', 'Quota exceeded for', 'Resource exhausted', the 'temporarily out of capacity' 429 text, or (a litellm-specific quirk) 'IndexError: list index out of range' — and litellm maps it to RateLimitError with a synthetic 429 response. All of these mean back off: a project quota is exhausted or Vertex is temporarily out of capacity.

Source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:1168

            model=model,
            llm_provider=custom_llm_provider,
            litellm_debug_info=extra_information,
            response=httpx.Response(
                status_code=400,
                request=httpx.Request(
                    method="POST",
                    url=" https://cloud.google.com/vertex-ai/",
                ),
            ),
        )
    elif (
        "429 Quota exceeded" in error_str
        or "Quota exceeded for" in error_str
        or "Resource exhausted" in error_str
        or "IndexError: list index out of range" in error_str
        or "429 Unable to submit request because the service is temporarily out of capacity." in error_str
    ):
        raise RateLimitError(
            message=f"litellm.RateLimitError: {custom_llm_provider}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            litellm_debug_info=extra_information,
            response=httpx.Response(
                status_code=429,
                request=httpx.Request(
                    method="POST",
                    url=" https://cloud.google.com/vertex-ai/",
                ),
            ),
        )
    elif (
        isinstance(getattr(original_exception, "status_code", None), int)
        and 500 <= original_exception.status_code < 600
        and _get_body_error_code(error_str) == 429
    ):
        # upstream gateway wraps a 429 inside a 5xx envelope

View on GitHub (pinned to 77b7c6c40c)

Solutions

  1. Retry with exponential backoff honoring retry-after (litellm num_retries, or Router with cooldown_time)
  2. Request a quota increase in the GCP console (IAM & Admin -> Quotas) for the specific model and region
  3. Spread traffic across regions or projects to side-step a single quota
  4. Cap client concurrency and batch requests to stay under RPM/TPM limits

Example fix

# before
for prompt in prompts:
    litellm.completion(model='vertex_ai/gemini-1.5-flash', messages=[{'role': 'user', 'content': prompt}])
# burst -> Quota exceeded

# after
for prompt in prompts:
    litellm.completion(
        model='vertex_ai/gemini-1.5-flash',
        messages=[{'role': 'user', 'content': prompt}],
        num_retries=5,
    )
# plus: request RPM/TPM quota increase for the region
Defensive patterns

Strategy: retry

Validate before calling

import time

class TokenBucket:
    def __init__(self, rpm: int):
        self.interval = 60.0 / rpm
        self._last = 0.0
    def acquire(self):
        wait = self._last + self.interval - time.monotonic()
        if wait > 0:
            time.sleep(wait)
        self._last = time.monotonic()

bucket = TokenBucket(rpm=project_quota_rpm)  # acquired from GCP quota API
bucket.acquire()  # before each vertex call

Type guard

import litellm

def is_vertex_quota_error(exc: BaseException) -> bool:
    msg = str(exc)
    return isinstance(exc, litellm.exceptions.RateLimitError) and any(
        s in msg for s in ('Quota exceeded', 'Resource exhausted', 'out of capacity')
    )

Try / catch

import litellm
from litellm.exceptions import RateLimitError

try:
    resp = litellm.completion(model='vertex_ai/gemini-1.5-flash', messages=messages)
except RateLimitError:
    # litellm honors Retry-After with num_retries; otherwise back off manually
    time.sleep(30)
    resp = litellm.completion(model='vertex_ai/gemini-1.5-flash', messages=messages, num_retries=5)

Prevention

When it happens

Trigger: Hitting per-minute RPM/TPM quotas for the model+region (e.g. gemini free-tier RPM limits); burst traffic on a project with low default quotas; regional capacity shortage for a newly released model returning the out-of-capacity 429 text.

Common situations: Fresh GCP projects with low default quotas; load or eval tests bursting requests; shared org quotas consumed by other teams; the IndexError string quirk making an unrelated client crash look like a quota error.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/497b8111caeab1db. Report an issue: GitHub.