BerriAI/litellm · error · RateLimitError

litellm.RateLimitError: {custom_llm_provider.capitalize()}Ex

Error message

litellm.RateLimitError: {custom_llm_provider.capitalize()}Exception - {error_str}

What it means

The status-based rate-limit branch: if the Vertex exception's status_code is exactly 429, LiteLLM raises RateLimitError with a synthetic 429 response and litellm_debug_info attached. This is the straightforward quota/throughput signal (RESOURCE_EXHAUSTED with HTTP 429), distinct from the string-matched quota phrases earlier in the chain. LiteLLM's Router treats it as a cooldown+retry candidate.

Source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:1258

                        url="https://cloud.google.com/vertex-ai/",
                    ),
                ),
            )
        if original_exception.status_code == 404:
            raise NotFoundError(
                message=f"{custom_llm_provider.capitalize()}Exception - {error_str}",
                llm_provider=custom_llm_provider,
                model=model,
            )
        if original_exception.status_code == 408:
            raise Timeout(
                message=f"{custom_llm_provider.capitalize()}Exception - {error_str}",
                llm_provider=custom_llm_provider,
                model=model,
            )

        if original_exception.status_code == 429:
            raise RateLimitError(
                message=f"litellm.RateLimitError: {custom_llm_provider.capitalize()}Exception - {error_str}",
                model=model,
                llm_provider=custom_llm_provider,
                litellm_debug_info=extra_information,
                response=httpx.Response(
                    status_code=429,
                    request=httpx.Request(
                        method="POST",
                        url=" https://cloud.google.com/vertex-ai/",
                    ),
                ),
            )
        if original_exception.status_code == 500:
            raise litellm.InternalServerError(
                message=f"{custom_llm_provider.capitalize()}Exception InternalServerError - {error_str}",
                model=model,
                llm_provider=custom_llm_provider,
                litellm_debug_info=extra_information,

View on GitHub (pinned to 6c2dcb801b)

Solutions

Retry with exponential backoff honoring Retry-Info: completion(..., num_retries=5) or Router with cooldown_time
Cap concurrency and rate: Router rpm/tpm limits or a client-side limiter under the quota
Request quota increases in GCP (IAM & Admin -> Quotas) for the specific model
Add fallback deployments/regions via Router rate_limit_error_fallbacks

Example fix

# before
import asyncio
await asyncio.gather(*[
    acompletion(model="vertex_ai/gemini-1.5-pro", messages=[m]) for m in many
])
# RateLimitError - 429

# after: bounded concurrency + retries
import asyncio
from litellm import acompletion
SEM = asyncio.Semaphore(5)
async def one(m):
    async with SEM:
        return await acompletion(
            model="vertex_ai/gemini-1.5-pro", messages=[m], num_retries=5,
        )
results = await asyncio.gather(*[one(m) for m in many])

Defensive patterns

Strategy: retry

Try / catch

import litellm, asyncio, random

async def complete_with_backoff(msgs, tries=5):
    for attempt in range(tries):
        try:
            return await litellm.acompletion(model="vertex_ai/gemini-1.5-pro", messages=msgs)
        except litellm.RateLimitError:
            await asyncio.sleep(min(2 ** attempt + random.random(), 120))
    raise litellm.RateLimitError(message="exhausted retries", model="x", llm_provider="x")

Prevention

Set client rpm/tpm 10-20% under the GCP quota
Honor retry-after hints from the error body
Provision quota increases ahead of traffic growth, not after 429 pages

When it happens

Trigger: vertex_ai completion calls exceeding PerModelPerProject RPM/TPM quota, provisioned-throughput limits, or a fresh project's default low quota — Google returns HTTP 429 and LiteLLM maps it directly.

Common situations: Fan-out batch jobs with high parallelism; default quotas on brand-new GCP projects (often single-digit QPM for big models); multi-team shared project quota exhaustion; bursts after a deploy warming up many workers.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/895a3cdcb10c54ef. Report an issue: GitHub.