BerriAI/litellm · error · ContextWindowExceededError

ContextWindowExceededError: {custom_llm_provider.capitalize(

Error message

ContextWindowExceededError: {custom_llm_provider.capitalize()}Exception - {error_str}

What it means

This is the generic context-window branch of _map_vertex_exception: when ExceptionCheckers.is_error_str_context_window_exceeded(error_str) matches (e.g. Google's 'input tokens exceed the model's context window' style messages), LiteLLM raises ContextWindowExceededError. It signals the request's token count is larger than the model's maximum input context. Callers use it to drive truncation or fallback to a larger-context model.

Source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:1105

            model=model,
            llm_provider=custom_llm_provider,
            response=httpx.Response(
                status_code=400,
                request=httpx.Request(
                    method="POST",
                    url=" https://cloud.google.com/vertex-ai/",
                ),
            ),
            litellm_debug_info=extra_information,
        )
    if "400 Request payload size exceeds" in error_str:
        raise ContextWindowExceededError(
            message=f"{custom_llm_provider.capitalize()}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
        )
    elif ExceptionCheckers.is_error_str_context_window_exceeded(error_str):
        raise ContextWindowExceededError(
            message=f"ContextWindowExceededError: {custom_llm_provider.capitalize()}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            litellm_debug_info=extra_information,
        )
    elif "None Unknown Error." in error_str or "Content has no parts." in error_str:
        raise litellm.InternalServerError(
            message=f"litellm.InternalServerError: {custom_llm_provider}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            response=httpx.Response(
                status_code=500,
                content=str(original_exception),
                request=httpx.Request(method="completion", url="https://github.com/BerriAI/litellm"),
            ),
            litellm_debug_info=extra_information,
        )
    elif "API key not valid." in error_str:

View on GitHub (pinned to 6c2dcb801b)

Solutions

Count tokens before sending (litellm.token_counter / model_info context limits) and truncate or summarize the input
Switch to a larger-context model, e.g. gemini-1.5-pro or gemini-1.5-flash-8b with 1M context
Enable prompt compression or use litellm's context-window-fallbacks in Router to auto-fallback to a bigger model
Summarize older conversation turns instead of replaying the full history

Example fix

# before
resp = completion(model="vertex_ai/gemini-1.5-flash-8b", messages=big_history)
# ContextWindowExceededError

# after: check and truncate first
from litellm import token_counter
MAX_IN = 1_000_000
while big_history and token_counter(model="vertex_ai/gemini-1.5-flash-8b", messages=big_history) > MAX_IN:
    big_history.pop(1)  # drop oldest non-system turn
resp = completion(model="vertex_ai/gemini-1.5-flash-8b", messages=big_history)

Defensive patterns

Strategy: validation

Validate before calling

from litellm import token_counter

def fits_context(model: str, messages: list, headroom: int = 512) -> bool:
    used = token_counter(model=model, messages=messages)
    max_in = litellm.get_max_input_tokens(model) if hasattr(litellm, 'get_max_input_tokens') else 1_000_000
    return used + headroom <= max_in

while not fits_context("vertex_ai/gemini-1.5-pro", msgs):
    msgs.pop(1)  # drop oldest non-system turn

Try / catch

import litellm

try:
    resp = litellm.completion(model="vertex_ai/gemini-1.5-pro", messages=msgs)
except litellm.ContextWindowExceededError:
    resp = litellm.completion(model="vertex_ai/gemini-1.5-pro-002", messages=msgs)  # 2M fallback

Prevention

Token-count before send and truncate/summarize proactively
Use Router context_window_fallbacks to auto-switch to a larger-context model
Track token usage per conversation and warn at 80% of the limit

When it happens

Trigger: A vertex_ai call where the total prompt tokens exceed the chosen model's context limit (e.g. >1M tokens for gemini-1.5-pro variants, >128k for others), producing an error string containing token-limit phrasing that the ExceptionCheckers heuristic recognizes.

Common situations: Whole-file or whole-repo prompts fed into a small-context model; conversation histories that grow past the limit mid-session; using gemini-1.5-flash (1M) sized prompts against a 128k model; not trimming tool-call transcripts in agents.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/558ad7d09b355947. Report an issue: GitHub.