BerriAI/litellm · error · ContextWindowExceededError

Exception

Error message

{custom_llm_provider.capitalize()}Exception - {error_str}

What it means

Vertex rejected the request with '400 Request payload size exceeds...' and litellm maps it to ContextWindowExceededError (message form '<Provider>Exception - <error_str>'). It fires when the raw request payload exceeds Vertex's request-size limit, which in practice happens with huge inline base64 media or genuinely oversized prompts.

Solutions

  1. Shrink the payload: downscale/compress images, trim documents, and send media via GCS URI where the model supports it instead of inline base64
  2. Chunk inputs and use map-reduce/summarize instead of one giant prompt
  3. Count tokens before sending: litellm.token_counter(model, messages=messages) and stay under the model limit in model_prices_and_context_window.json

Example fix

# before
litellm.completion(
    model='vertex_ai/gemini-1.5-pro',
    messages=[{'role': 'user', 'content': [
        {'type': 'image_url', 'image_url': {'url': 'data:image/png;base64,' + huge_b64}},  # 30 MB inline
        {'type': 'text', 'text': huge_doc},
    ]}],
)

# after
litellm.completion(
    model='vertex_ai/gemini-1.5-pro',
    messages=[{'role': 'user', 'content': [
        {'type': 'image_url', 'image_url': {'url': 'gs://my-bucket/img.png'}},  # by reference
        {'type': 'text', 'text': huge_doc[:100_000]},  # trimmed
    ]}],
)
Defensive patterns

Strategy: validation

Validate before calling

import base64

MAX_INLINE_BYTES = 10 * 1024 * 1024  # stay well under Vertex payload cap

def payload_fits(parts: list[dict]) -> bool:
    size = 0
    for p in parts:
        if p.get('type') == 'image_url':
            url = p['image_url']['url']
            if url.startswith('data:'):
                size += len(base64.b64decode(url.split(',', 1)[1]))
        else:
            size += len(p.get('text', ''))
    return size < MAX_INLINE_BYTES

assert payload_fits(content_parts), 'payload too large - reference media via gs:// instead'

Type guard

import litellm

def is_vertex_payload_too_large(exc: BaseException) -> bool:
    return isinstance(exc, litellm.exceptions.ContextWindowExceededError) and 'Request payload size exceeds' in str(exc)

Try / catch

import litellm
from litellm.exceptions import ContextWindowExceededError

try:
    resp = litellm.completion(model='vertex_ai/gemini-1.5-pro', messages=messages)
except ContextWindowExceededError as e:
    if 'Request payload size exceeds' in str(e):
        messages = downscale_inline_media(messages)  # compress or move to gs:// URIs
        resp = litellm.completion(model='vertex_ai/gemini-1.5-pro', messages=messages)
    raise

Prevention

When it happens

Trigger: Multimodal litellm calls passing large base64-encoded images/audio/video inline; very long prompt text pushed past Vertex's per-request payload cap; RAG pipelines stuffing too many retrieved chunks into one request.

Common situations: Whole documents pasted into prompts; high-resolution images sent inline instead of by reference; conversations that grew until the serialized body crossed the limit.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/c456bec7c19f2e08. Report an issue: GitHub.

Appendix: source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:1099

    exception_provider: str,
    extra_information: str,
) -> None:
    if "Vertex AI API has not been used in project" in error_str or "Unable to find your project" in error_str:
        raise BadRequestError(
            message=f"litellm.BadRequestError: {custom_llm_provider}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            response=httpx.Response(
                status_code=400,
                request=httpx.Request(
                    method="POST",
                    url=" https://cloud.google.com/vertex-ai/",
                ),
            ),
            litellm_debug_info=extra_information,
        )
    if "400 Request payload size exceeds" in error_str:
        raise ContextWindowExceededError(
            message=f"{custom_llm_provider.capitalize()}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
        )
    elif ExceptionCheckers.is_error_str_context_window_exceeded(error_str):
        raise ContextWindowExceededError(
            message=f"ContextWindowExceededError: {custom_llm_provider.capitalize()}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            litellm_debug_info=extra_information,
        )
    elif "None Unknown Error." in error_str or "Content has no parts." in error_str:
        raise litellm.InternalServerError(
            message=f"litellm.InternalServerError: {custom_llm_provider}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            response=httpx.Response(
                status_code=500,

View on GitHub (pinned to 77b7c6c40c)