BerriAI/litellm · error · ContextWindowExceededError

Exception: Context Window Error

Error message

{custom_llm_provider.capitalize()}Exception: Context Window Error - {error_str}

What it means

When ExceptionCheckers.is_error_str_context_window_exceeded matches the provider error text (an 'input is too long' style message from watsonx, databricks, predibase, or lemonade), litellm raises ContextWindowExceededError. It means the assembled prompt plus reserved completion tokens exceed the target model's context window. The litellm_debug_info field carries the provider's extra detail to help identify which part overflowed.

Solutions

1. Trim or summarize older messages before sending (sliding window or map-reduce)
2. Preflight with litellm.token_counter(model, messages=...) against the model's limit
3. Reduce max_tokens to leave room for the prompt
4. Switch to a variant with a larger context window

Example fix

# before - full history blows the window
resp = litellm.completion(model="watsonx/ibm/granite-3-8b-instruct", messages=all_messages)

# after - preflight token count, trim oldest turns
model = "watsonx/ibm/granite-3-8b-instruct"
limit = 4096
used = litellm.token_counter(model=model, messages=all_messages)
while used > limit - 512:  # reserve space for the answer
    all_messages = [all_messages[0]] + all_messages[2:]  # keep the system prompt
    used = litellm.token_counter(model=model, messages=all_messages)
resp = litellm.completion(model=model, messages=all_messages)

import litellm

def fits_context(model: str, messages: list[dict], max_tokens: int = 512) -> bool:
    used = litellm.token_counter(model=model, messages=messages)
    limit = litellm.get_max_tokens(model) or 4096
    return used + max_tokens <= limit

if not fits_context(model, messages):
    messages = condense(messages)

import litellm

def is_context_window_error(exc: BaseException) -> bool:
    return isinstance(exc, litellm.ContextWindowExceededError)

try:
    resp = litellm.completion(model=model, messages=messages)
except litellm.ContextWindowExceededError:
    messages = condense(messages)  # summarize older turns, keep system + latest
    resp = litellm.completion(model=model, messages=messages)

When it happens

Trigger: Long chat histories sent to a small-context watsonx model; RAG stuffing too many retrieved chunks; max_tokens large enough that prompt plus completion no longer fits the window.

Common situations: Conversation apps that accumulate messages forever; switching to a model with a smaller window without changing trimming logic; document QA that injects whole files.

Related errors

• AI21Exception
• AlephAlphaException
• ContextWindowExceededError
• TogetherAIException
• API key is required

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/8276ca3a4648acbd. Report an issue: GitHub.

Appendix: source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:725

    extra_information: str,
) -> None:
    if "authorization denied for" in error_str:
        # Predibase returns the raw API Key in the response - this block ensures it's not returned in the exception
        if error_str is not None and isinstance(error_str, str) and "bearer" in error_str.lower():
            # only keep the first 10 chars after the occurnence of "bearer"
            _bearer_token_start_index: Final = error_str.lower().find("bearer")
            error_str = error_str[: _bearer_token_start_index + 14]
            error_str += "XXXXXXX" + '"'

        raise AuthenticationError(
            message=f"{custom_llm_provider.capitalize()}Exception: Authentication Error - {error_str}",
            llm_provider=custom_llm_provider,
            model=model,
            response=getattr(original_exception, "response", None),
            litellm_debug_info=extra_information,
        )
    elif ExceptionCheckers.is_error_str_context_window_exceeded(error_str):
        raise ContextWindowExceededError(
            message=f"{custom_llm_provider.capitalize()}Exception: Context Window Error - {error_str}",
            model=model,
            llm_provider=custom_llm_provider,
            response=getattr(original_exception, "response", None),
            litellm_debug_info=extra_information,
        )
    elif "token_quota_reached" in error_str:
        raise RateLimitError(
            message=f"{custom_llm_provider.capitalize()}Exception: Rate Limit Errror - {error_str}",
            llm_provider=custom_llm_provider,
            model=model,
            response=getattr(original_exception, "response", None),
        )
    elif "The server received an invalid response from an upstream server." in error_str:
        raise litellm.InternalServerError(
            message=f"{custom_llm_provider.capitalize()}Exception - {original_exception.message}",
            llm_provider=custom_llm_provider,
            model=model,

View on GitHub (pinned to 77b7c6c40c)