BerriAI/litellm · error · ContextWindowExceededError

ContextWindowExceededError: {exception_provider} - {message}

Error message

ContextWindowExceededError: {exception_provider} - {message}

What it means

Normalized ContextWindowExceededError raised when the provider error string matches litellm's context-window heuristics (e.g. OpenAI's 'maximum context length is ... tokens, however you requested ...'). It signals the request's prompt (+ expected completion) exceeds the model's context window, not a transient failure — retrying unchanged will fail again.

Source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:298

            "openai.OpenAIError",
            f"{custom_llm_provider}.{custom_llm_provider}Error",
        )
    if custom_llm_provider == "openai":
        exception_provider = "OpenAI" + "Exception"
    else:
        exception_provider = custom_llm_provider[0].upper() + custom_llm_provider[1:] + "Exception"

    if ExceptionCheckers.is_error_str_rate_limit(
        error_str, status_code=getattr(original_exception, "status_code", None)
    ):
        raise RateLimitError(
            message=f"RateLimitError: {exception_provider} - {message}",
            model=model,
            llm_provider=custom_llm_provider,
            response=getattr(original_exception, "response", None),
        )
    elif ExceptionCheckers.is_error_str_context_window_exceeded(error_str):
        raise ContextWindowExceededError(
            message=f"ContextWindowExceededError: {exception_provider} - {message}",
            llm_provider=custom_llm_provider,
            model=model,
            response=getattr(original_exception, "response", None),
            litellm_debug_info=extra_information,
        )
    elif "invalid_request_error" in error_str and "model_not_found" in error_str:
        raise NotFoundError(
            message=f"{exception_provider} - {message}",
            llm_provider=custom_llm_provider,
            model=model,
            response=getattr(original_exception, "response", None),
            litellm_debug_info=extra_information,
        )
    elif "A timeout occurred" in error_str:
        raise Timeout(
            message=f"{exception_provider} - {message}",
            model=model,

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Trim or compress the prompt: drop old turns, summarize history, or chunk the document and map/reduce.
2. Switch the request to a model with a larger context window (or a provider long-context variant).
3. Reduce max_tokens so prompt+completion fits, and count tokens before sending (litellm's token_counter) to fail fast client-side.

Example fix

# before
resp = litellm.completion(model="gpt-3.5-turbo", messages=all_500k_tokens)
# -> ContextWindowExceededError

# after
from litellm import token_counter
msgs = trim_to_budget(all_messages, budget=6000)  # custom trim
assert token_counter(model="gpt-3.5-turbo", messages=msgs) < 8000
resp = litellm.completion(model="gpt-3.5-turbo", messages=msgs, max_tokens=1000)

from litellm import token_counter

def fits_context(model: str, messages: list, max_tokens: int) -> bool:
    limit = litellm.get_max_tokens(model) or 0
    return token_counter(model=model, messages=messages) + max_tokens < limit

if not fits_context(model, msgs, 1000):
    msgs = trim_history(msgs, budget=4000)

from litellm import ContextWindowExceededError

def is_context_window_error(exc: BaseException) -> bool:
    return isinstance(exc, ContextWindowExceededError)

from litellm import ContextWindowExceededError

try:
    resp = litellm.completion(model=m, messages=msgs)
except ContextWindowExceededError:
    msgs = summarize_history(msgs)          # shrink, then ONE retry — never loop unchanged
    resp = litellm.completion(model=m, messages=msgs)

When it happens

Trigger: Sending a long conversation or large document to a model whose window is smaller than prompt+max_tokens; vision/file payloads inflating token counts; provider strings like 'context_length_exceeded' hitting the is_error_str_context_window_exceeded branch.

Common situations: Summarizing big PDFs/transcripts on an 8k model; unbounded chat histories growing past the window over a long session; setting max_tokens close to the window leaving no room for the prompt; model downgrades (to a smaller-context variant) without trimming inputs.

Related errors

• {exception_provider} - {message}
• ContentPolicyViolationError: {exception_provider} - {message
• {custom_llm_provider.capitalize()}Exception: Context Window
• BedrockException: Context Window Error - {error_str}
• SagemakerException - {error_str}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/b40a8a4f46dbeb49. Report an issue: GitHub.