{"record":{"id":"558ad7d09b355947","repo":"BerriAI/litellm","slug":"contextwindowexceedederror-custom-llm-provider-c","errorCode":null,"errorMessage":"ContextWindowExceededError: {custom_llm_provider.capitalize()}Exception - {error_str}","messagePattern":"ContextWindowExceededError: (.+?)Exception - (.+?)","errorType":"exception","errorClass":"ContextWindowExceededError","httpStatus":400,"severity":"error","filePath":"litellm/litellm_core_utils/exception_mapping_utils.py","lineNumber":1105,"sourceCode":"            model=model,\n            llm_provider=custom_llm_provider,\n            response=httpx.Response(\n                status_code=400,\n                request=httpx.Request(\n                    method=\"POST\",\n                    url=\" https://cloud.google.com/vertex-ai/\",\n                ),\n            ),\n            litellm_debug_info=extra_information,\n        )\n    if \"400 Request payload size exceeds\" in error_str:\n        raise ContextWindowExceededError(\n            message=f\"{custom_llm_provider.capitalize()}Exception - {error_str}\",\n            model=model,\n            llm_provider=custom_llm_provider,\n        )\n    elif ExceptionCheckers.is_error_str_context_window_exceeded(error_str):\n        raise ContextWindowExceededError(\n            message=f\"ContextWindowExceededError: {custom_llm_provider.capitalize()}Exception - {error_str}\",\n            model=model,\n            llm_provider=custom_llm_provider,\n            litellm_debug_info=extra_information,\n        )\n    elif \"None Unknown Error.\" in error_str or \"Content has no parts.\" in error_str:\n        raise litellm.InternalServerError(\n            message=f\"litellm.InternalServerError: {custom_llm_provider}Exception - {error_str}\",\n            model=model,\n            llm_provider=custom_llm_provider,\n            response=httpx.Response(\n                status_code=500,\n                content=str(original_exception),\n                request=httpx.Request(method=\"completion\", url=\"https://github.com/BerriAI/litellm\"),\n            ),\n            litellm_debug_info=extra_information,\n        )\n    elif \"API key not valid.\" in error_str:","sourceCodeStart":1087,"sourceCodeEnd":1123,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/litellm_core_utils/exception_mapping_utils.py#L1087-L1123","documentation":"litellm ran the provider error string through ExceptionCheckers.is_error_str_context_window_exceeded, it matched a known context-window phrase, and litellm raises ContextWindowExceededError with the 'ContextWindowExceededError:' prefixed message. This is the generic Vertex path for prompts that exceed the model's token limit.","triggerScenarios":"input tokens (+ expected output) exceed the model's context window — e.g. sending 40k tokens to a 32k model; Vertex error text such as 'input tokens exceed maximum' or 'request exceeds maximum number of tokens' matches the checker.","commonSituations":"Conversation history growing across turns until it crosses the limit; system prompt plus few-shot examples eating the budget; assuming a larger context than the model variant actually has (flash vs pro limits).","solutions":["Count tokens before the call and truncate: litellm.token_counter(model, messages=messages)","Trim oldest turns or summarize the conversation when over budget","Switch to a larger-context model variant (e.g. gemini-1.5-pro-002 instead of a smaller-context model)","Cap max_tokens so prompt + completion fit inside the window"],"exampleFix":"# before\nresponse = litellm.completion(\n    model='vertex_ai/gemini-1.5-flash',\n    messages=full_history,  # grew past the context limit\n)\n\n# after\nfrom litellm import token_counter\nMAX_INPUT = 900_000\nif token_counter(model='vertex_ai/gemini-1.5-flash', messages=full_history) > MAX_INPUT:\n    full_history = full_history[:1] + full_history[-10:]  # keep system + recent turns\nresponse = litellm.completion(\n    model='vertex_ai/gemini-1.5-flash',\n    messages=full_history,\n)","handlingStrategy":"validation","validationCode":"import litellm\n\ndef fits_context(model: str, messages: list[dict], max_tokens: int) -> bool:\n    used = litellm.token_counter(model=model, messages=messages)\n    limit = litellm.get_max_tokens(model)\n    return used + max_tokens < (limit or float('inf'))\n\nif not fits_context('vertex_ai/gemini-1.5-flash', messages, max_tokens=1024):\n    messages = trim_history(messages, keep_last=10)","typeGuard":"import litellm\n\ndef is_vertex_context_exceeded(exc: BaseException) -> bool:\n    return isinstance(exc, litellm.exceptions.ContextWindowExceededError) and 'ContextWindowExceededError' in str(exc)","tryCatchPattern":"import litellm\nfrom litellm.exceptions import ContextWindowExceededError\n\ntry:\n    resp = litellm.completion(model='vertex_ai/gemini-1.5-flash', messages=messages)\nexcept ContextWindowExceededError:\n    messages = summarize_older_turns(messages)\n    resp = litellm.completion(model='vertex_ai/gemini-1.5-flash', messages=messages)","preventionTips":["Run litellm.token_counter before every call with dynamic history","Trim or summarize conversation history on a token budget, not a message count","Read each model's context limit from litellm.get_max_tokens instead of assuming"],"tags":["vertex-ai","gcp","context-window","token-limit","litellm"],"backgroundTag":"context-window-exceeded","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}