BerriAI/litellm · error · ContextWindowExceededError

ContextWindowExceededError

Error message

ContextWindowExceededError: {custom_llm_provider.capitalize()}Exception - {error_str}

What it means

litellm ran the provider error string through ExceptionCheckers.is_error_str_context_window_exceeded, it matched a known context-window phrase, and litellm raises ContextWindowExceededError with the 'ContextWindowExceededError:' prefixed message. This is the generic Vertex path for prompts that exceed the model's token limit.

Solutions

1. Count tokens before the call and truncate: litellm.token_counter(model, messages=messages)
2. Trim oldest turns or summarize the conversation when over budget
3. Switch to a larger-context model variant (e.g. gemini-1.5-pro-002 instead of a smaller-context model)
4. Cap max_tokens so prompt + completion fit inside the window

Example fix

# before
response = litellm.completion(
    model='vertex_ai/gemini-1.5-flash',
    messages=full_history,  # grew past the context limit
)

# after
from litellm import token_counter
MAX_INPUT = 900_000
if token_counter(model='vertex_ai/gemini-1.5-flash', messages=full_history) > MAX_INPUT:
    full_history = full_history[:1] + full_history[-10:]  # keep system + recent turns
response = litellm.completion(
    model='vertex_ai/gemini-1.5-flash',
    messages=full_history,
)

import litellm

def fits_context(model: str, messages: list[dict], max_tokens: int) -> bool:
    used = litellm.token_counter(model=model, messages=messages)
    limit = litellm.get_max_tokens(model)
    return used + max_tokens < (limit or float('inf'))

if not fits_context('vertex_ai/gemini-1.5-flash', messages, max_tokens=1024):
    messages = trim_history(messages, keep_last=10)

import litellm

def is_vertex_context_exceeded(exc: BaseException) -> bool:
    return isinstance(exc, litellm.exceptions.ContextWindowExceededError) and 'ContextWindowExceededError' in str(exc)

import litellm
from litellm.exceptions import ContextWindowExceededError

try:
    resp = litellm.completion(model='vertex_ai/gemini-1.5-flash', messages=messages)
except ContextWindowExceededError:
    messages = summarize_older_turns(messages)
    resp = litellm.completion(model='vertex_ai/gemini-1.5-flash', messages=messages)

When it happens

Trigger: input tokens (+ expected output) exceed the model's context window — e.g. sending 40k tokens to a 32k model; Vertex error text such as 'input tokens exceed maximum' or 'request exceeds maximum number of tokens' matches the checker.

Common situations: Conversation history growing across turns until it crosses the limit; system prompt plus few-shot examples eating the budget; assuming a larger context than the model variant actually has (flash vs pro limits).

Related errors

• Exception
• AI21Exception
• AlephAlphaException
• Exception BadRequestError
• Exception: Context Window Error

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/558ad7d09b355947. Report an issue: GitHub.

Appendix: source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:1105

            model=model,
            llm_provider=custom_llm_provider,
            response=httpx.Response(
                status_code=400,
                request=httpx.Request(
                    method="POST",
                    url=" https://cloud.google.com/vertex-ai/",
                ),
            ),
            litellm_debug_info=extra_information,
        )
    if "400 Request payload size exceeds" in error_str:
        raise ContextWindowExceededError(
            message=f"{custom_llm_provider.capitalize()}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
        )
    elif ExceptionCheckers.is_error_str_context_window_exceeded(error_str):
        raise ContextWindowExceededError(
            message=f"ContextWindowExceededError: {custom_llm_provider.capitalize()}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            litellm_debug_info=extra_information,
        )
    elif "None Unknown Error." in error_str or "Content has no parts." in error_str:
        raise litellm.InternalServerError(
            message=f"litellm.InternalServerError: {custom_llm_provider}Exception - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            response=httpx.Response(
                status_code=500,
                content=str(original_exception),
                request=httpx.Request(method="completion", url="https://github.com/BerriAI/litellm"),
            ),
            litellm_debug_info=extra_information,
        )
    elif "API key not valid." in error_str:

View on GitHub (pinned to 77b7c6c40c)