BerriAI/litellm · error · ContextWindowExceededError

BedrockException: Context Window Error

Error message

BedrockException: Context Window Error - {error_str}

What it means

LiteLLM throws ContextWindowExceededError (a BadRequestError subclass, HTTP 400) when AWS Bedrock rejects a request whose input exceeds the model's context window. The mapper at litellm/litellm_core_utils/exception_mapping_utils.py:832 matches the provider strings ('too many tokens', 'expected maxLength:', 'Input is too long', 'prompt is too long', 'prompt: length: 1..', 'Too many input tokens') from Titan, Anthropic-on-Bedrock, and Cohere, and re-raises them as one normalized type so cross-provider code needs a single except clause. Retrying the identical payload always fails again: the input must shrink or the window must grow.

Solutions

1. Shrink the input: drop oldest messages, cut retrieved chunk count, or summarize the context until under the limit
2. Pre-flight the budget with litellm.get_max_tokens(model) and litellm.token_counter(model=..., messages=...) and cap max_tokens to (window - input - margin)
3. Switch to a Bedrock model variant with a larger context window (Claude / Titan Premier instead of Titan Express)
4. In the proxy, trim requests via litellm_settings (e.g. history caps / drop_params) so oversized payloads never reach Bedrock

Example fix

# before
resp = litellm.completion(model="bedrock/amazon.titan-text-express-v1", messages=all_messages, max_tokens=1024)

# after
limit = litellm.get_max_tokens("bedrock/amazon.titan-text-express-v1")
while litellm.token_counter(model="bedrock/amazon.titan-text-express-v1", messages=all_messages) + 1024 > limit - 64:
    all_messages = [all_messages[0]] + all_messages[2:]  # drop oldest turn, keep system
resp = litellm.completion(model="bedrock/amazon.titan-text-express-v1", messages=all_messages, max_tokens=1024)

import litellm

def fits_context(model: str, messages: list, max_tokens: int, margin: int = 64) -> bool:
    limit = litellm.get_max_tokens(model) or 0
    used = litellm.token_counter(model=model, messages=messages) + max_tokens
    return used <= limit - margin

import litellm

def is_context_window_error(e: BaseException) -> bool:
    return isinstance(e, litellm.ContextWindowExceededError)

try:
    resp = litellm.completion(model=model, messages=messages)
except litellm.ContextWindowExceededError:
    messages = [messages[0]] + messages[2:]  # drop oldest turn, keep system
    resp = litellm.completion(model=model, messages=messages)  # retry once, trimmed

When it happens

Trigger: litellm.completion(model='bedrock/...', messages=[...]) where prompt tokens (or prompt + max_tokens) exceed the deployed model's limit: Amazon Titan returns 'Input is too long', Anthropic on Bedrock returns 'prompt is too long' / 'expected maxLength:', Cohere returns 'Too many input tokens'. Long chat histories accumulated across agent turns are the most common producer.

Common situations: RAG pipelines stuffing many retrieved chunks into one prompt; sending whole files to a small-window model (Titan Express is 8k); unbounded chat history growth in chat apps; budgeting only prompt tokens and forgetting that max_tokens counts against the same window.

Related errors

• AI21Exception
• AlephAlphaException
• Exception: Context Window Error
• AgentCore: Failed to read/parse JSON response body
• Amazon Nova Canvas INPAINTING requires either maskPrompt or…

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/395e524d24fb1ff3. Report an issue: GitHub.

Appendix: source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:832

def _map_bedrock_exception(
    *,
    model: str,
    original_exception: _ProviderHTTPException,
    custom_llm_provider: str,
    error_str: str,
    exception_type: str,
    exception_provider: str,
    extra_information: str,
) -> None:
    if (
        "too many tokens" in error_str
        or "expected maxLength:" in error_str
        or "Input is too long" in error_str
        or "prompt is too long" in error_str
        or "prompt: length: 1.." in error_str
        or "Too many input tokens" in error_str
    ):
        raise ContextWindowExceededError(
            message=f"BedrockException: Context Window Error - {error_str}",
            model=model,
            llm_provider="bedrock",
        )
    elif "Conversation blocks and tool result blocks cannot be provided in the same turn." in error_str:
        raise BadRequestError(
            message=f"BedrockException - {error_str}\n. Enable 'litellm.modify_params=True' (for PROXY do: `litellm_settings::modify_params: True`) to insert a dummy assistant message and fix this error.",
            model=model,
            llm_provider="bedrock",
            response=getattr(original_exception, "response", None),
        )
    elif "Malformed input request" in error_str:
        raise BadRequestError(
            message=f"BedrockException - {error_str}",
            model=model,
            llm_provider="bedrock",
            response=getattr(original_exception, "response", None),
        )

View on GitHub (pinned to 77b7c6c40c)