BerriAI/litellm · error · ContextWindowExceededError

AzureException ContextWindowExceededError - {message}

Error message

AzureException ContextWindowExceededError - {message}

What it means

litellm maps Azure OpenAI errors containing "This model's maximum context length is" to litellm.ContextWindowExceededError. Prompt tokens plus requested completion exceed the deployed Azure model's context window.

Source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:1905

                if azure_error_code != "content_policy_violation":
                    _inner: Final = body_dict["error"].get("inner_error") or body_dict["error"].get("innererror")
                    if isinstance(_inner, dict) and _inner.get("code") == "ResponsibleAIPolicyViolation":
                        azure_error_code = "content_policy_violation"
            else:
                azure_error_code = body_dict.get("code")
    except Exception:
        azure_error_code = None

    if "Internal server error" in error_str:
        raise litellm.InternalServerError(
            message=f"AzureException Internal server error - {message}",
            llm_provider="azure",
            model=model,
            litellm_debug_info=extra_information,
            response=getattr(original_exception, "response", None),
        )
    elif "This model's maximum context length is" in error_str:
        raise ContextWindowExceededError(
            message=f"AzureException ContextWindowExceededError - {message}",
            llm_provider="azure",
            model=model,
            litellm_debug_info=extra_information,
            response=getattr(original_exception, "response", None),
        )
    elif "DeploymentNotFound" in error_str:
        raise NotFoundError(
            message=f"AzureException NotFoundError - {message}",
            llm_provider="azure",
            model=model,
            litellm_debug_info=extra_information,
            response=getattr(original_exception, "response", None),
        )
    elif azure_error_code == "content_policy_violation" or ExceptionCheckers.is_azure_content_policy_violation_error(
        error_str
    ):
        from litellm.llms.azure.exception_mapping import (

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Trim messages/summaries so prompt + max_tokens fit the deployed model's window
2. Point the call at a deployment of a larger-context model version (e.g. gpt-35-turbo-16k, gpt-4o)
3. Pre-count tokens with litellm.token_counter(model=<azure model>) and enforce a budget
4. Cap conversation history length with litellm Router's trim messages helpers

Example fix

# before
litellm.completion(model='azure/my-gpt35-deployment', messages=long_rag_msgs, max_tokens=2000)
# after
limit = 16384 if '16k' in deployment_version else 4097
msgs = trim_messages_to(long_rag_msgs, limit - 2000 - 64)
litellm.completion(model='azure/my-gpt35-16k-deployment', messages=msgs, max_tokens=2000)

used = litellm.token_counter(model='gpt-3.5-turbo', messages=msgs) + max_tokens
limit = 4097  # per deployed version
if used > limit:
    msgs = trim_to_budget(msgs, limit - max_tokens)

import litellm

def is_azure_context_overflow(e: Exception) -> bool:
    return isinstance(e, litellm.ContextWindowExceededError) and 'azure' in str(getattr(e, 'llm_provider', ''))

try:
    litellm.completion(...)
except litellm.ContextWindowExceededError:
    msgs = trim_to_budget(msgs)
    return litellm.completion(..., messages=msgs)

When it happens

Trigger: An Azure deployment call (model='azure/...') whose prompt plus max_tokens exceeds the underlying model's limit (e.g. 4k/8k/32k/128k depending on version); Azure returns the context-length error and litellm converts it, attaching litellm_debug_info.

Common situations: Long RAG contexts stuffed into gpt-35-turbo (4k) deployments; conversation histories that grow past the window mid-session; assuming 16k/32k limits on a 4k deployment SKU.

Related errors

• AlephAlphaException - {original_exception.message}
• soft_budget cannot be negative. Received: {data.soft_budget}
• soft_budget ({data.soft_budget}) must be strictly lower than
• Model '{m}' not in team's allowed models. Team allowed model
• Project max_budget ({data.max_budget}) exceeds team's max_bu

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/70462b762bc6ed88. Report an issue: GitHub.