BerriAI/litellm · error · ContextWindowExceededError

{custom_llm_provider.capitalize()}Exception - {error_str}

Error message

{custom_llm_provider.capitalize()}Exception - {error_str}

What it means

LiteLLM maps a Vertex AI '400 Request payload size exceeds' error to ContextWindowExceededError. Google rejects the request body because it exceeds the payload size limit (roughly 20 MB per request for the Vertex AI endpoint), which in practice happens when the prompt plus inline data (images, audio, video) is too large. LiteLLM classifies it as a context-window problem so existing context-window retry logic (truncation, fallback models) applies.

Source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:1099

    exception_provider: str,
    extra_information: str,
) -> None:
    if "Vertex AI API has not been used in project" in error_str or "Unable to find your project" in error_str:
        raise BadRequestError(
            message=f"litellm.BadRequestError: {custom_llm_provider}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            response=httpx.Response(
                status_code=400,
                request=httpx.Request(
                    method="POST",
                    url=" https://cloud.google.com/vertex-ai/",
                ),
            ),
            litellm_debug_info=extra_information,
        )
    if "400 Request payload size exceeds" in error_str:
        raise ContextWindowExceededError(
            message=f"{custom_llm_provider.capitalize()}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
        )
    elif ExceptionCheckers.is_error_str_context_window_exceeded(error_str):
        raise ContextWindowExceededError(
            message=f"ContextWindowExceededError: {custom_llm_provider.capitalize()}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            litellm_debug_info=extra_information,
        )
    elif "None Unknown Error." in error_str or "Content has no parts." in error_str:
        raise litellm.InternalServerError(
            message=f"litellm.InternalServerError: {custom_llm_provider}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            response=httpx.Response(
                status_code=500,

View on GitHub (pinned to 6c2dcb801b)

Solutions

  1. Reduce the payload: truncate the message history (keep the system prompt plus last N turns) before sending
  2. Move large media out of the request: upload files via the Vertex AI Files API or GCS and pass a file_uri instead of inline base64
  3. Compress or downsample images/audio before inlining them
  4. Switch to a model/endpoint that accepts larger payloads, or split the document into chunked requests

Example fix

# before
resp = completion(
    model="vertex_ai/gemini-1.5-pro",
    messages=messages,  # 40 MB of base64 images + full history
)

# after: truncate history and reference uploaded files
trim = max(1, len(messages) - 20)
resp = completion(
    model="vertex_ai/gemini-1.5-pro",
    messages=[messages[0]] + messages[trim:],  # system + last 20 turns
    # media uploaded via Files API -> file_uri, not base64 inline
)
Defensive patterns

Strategy: validation

Validate before calling

import sys

def payload_too_large(messages, limit=20 * 1024 * 1024) -> bool:
    """Approximate serialized request size before sending."""
    size = sys.getsizeof(repr(messages))
    return size > limit, size

too_big, size = payload_too_large(messages)
if too_big:
    messages = [messages[0]] + messages[-20:]  # trim history
    # or move media to Files API / GCS file_uri references

Try / catch

import litellm

try:
    resp = litellm.completion(model="vertex_ai/gemini-1.5-pro", messages=messages)
except litellm.ContextWindowExceededError:
    messages = [messages[0]] + messages[-20:]      # truncate and retry once
    resp = litellm.completion(model="vertex_ai/gemini-1.5-pro", messages=messages)

Prevention

When it happens

Trigger: A vertex_ai completion call whose serialized JSON payload (messages, base64-encoded images/files, long conversation history) exceeds Vertex AI's request payload limit; typically multimodal requests that inline large media or chat histories with many accumulated messages.

Common situations: Sending base64 images or PDFs inline that blow past the payload cap; long agent conversations where the message list grows unboundedly; pasting large documents into the prompt instead of using the Vertex Files API; caching disabled so full context is re-sent every turn.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/c456bec7c19f2e08. Report an issue: GitHub.