{"record":{"id":"558ad7d09b355947","repo":"BerriAI/litellm","slug":"contextwindowexceedederror-custom-llm-provider-c","errorCode":null,"errorMessage":"ContextWindowExceededError: {custom_llm_provider.capitalize()}Exception - {error_str}","messagePattern":"ContextWindowExceededError: (.+?)Exception - (.+?)","errorType":"exception","errorClass":"ContextWindowExceededError","httpStatus":400,"severity":"error","filePath":"litellm/litellm_core_utils/exception_mapping_utils.py","lineNumber":1105,"sourceCode":"            model=model,\n            llm_provider=custom_llm_provider,\n            response=httpx.Response(\n                status_code=400,\n                request=httpx.Request(\n                    method=\"POST\",\n                    url=\" https://cloud.google.com/vertex-ai/\",\n                ),\n            ),\n            litellm_debug_info=extra_information,\n        )\n    if \"400 Request payload size exceeds\" in error_str:\n        raise ContextWindowExceededError(\n            message=f\"{custom_llm_provider.capitalize()}Exception - {error_str}\",\n            model=model,\n            llm_provider=custom_llm_provider,\n        )\n    elif ExceptionCheckers.is_error_str_context_window_exceeded(error_str):\n        raise ContextWindowExceededError(\n            message=f\"ContextWindowExceededError: {custom_llm_provider.capitalize()}Exception - {error_str}\",\n            model=model,\n            llm_provider=custom_llm_provider,\n            litellm_debug_info=extra_information,\n        )\n    elif \"None Unknown Error.\" in error_str or \"Content has no parts.\" in error_str:\n        raise litellm.InternalServerError(\n            message=f\"litellm.InternalServerError: {custom_llm_provider}Exception - {error_str}\",\n            model=model,\n            llm_provider=custom_llm_provider,\n            response=httpx.Response(\n                status_code=500,\n                content=str(original_exception),\n                request=httpx.Request(method=\"completion\", url=\"https://github.com/BerriAI/litellm\"),\n            ),\n            litellm_debug_info=extra_information,\n        )\n    elif \"API key not valid.\" in error_str:","sourceCodeStart":1087,"sourceCodeEnd":1123,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/litellm_core_utils/exception_mapping_utils.py#L1087-L1123","documentation":"This is the generic context-window branch of _map_vertex_exception: when ExceptionCheckers.is_error_str_context_window_exceeded(error_str) matches (e.g. Google's 'input tokens exceed the model's context window' style messages), LiteLLM raises ContextWindowExceededError. It signals the request's token count is larger than the model's maximum input context. Callers use it to drive truncation or fallback to a larger-context model.","triggerScenarios":"A vertex_ai call where the total prompt tokens exceed the chosen model's context limit (e.g. >1M tokens for gemini-1.5-pro variants, >128k for others), producing an error string containing token-limit phrasing that the ExceptionCheckers heuristic recognizes.","commonSituations":"Whole-file or whole-repo prompts fed into a small-context model; conversation histories that grow past the limit mid-session; using gemini-1.5-flash (1M) sized prompts against a 128k model; not trimming tool-call transcripts in agents.","solutions":["Count tokens before sending (litellm.token_counter / model_info context limits) and truncate or summarize the input","Switch to a larger-context model, e.g. gemini-1.5-pro or gemini-1.5-flash-8b with 1M context","Enable prompt compression or use litellm's context-window-fallbacks in Router to auto-fallback to a bigger model","Summarize older conversation turns instead of replaying the full history"],"exampleFix":"# before\nresp = completion(model=\"vertex_ai/gemini-1.5-flash-8b\", messages=big_history)\n# ContextWindowExceededError\n\n# after: check and truncate first\nfrom litellm import token_counter\nMAX_IN = 1_000_000\nwhile big_history and token_counter(model=\"vertex_ai/gemini-1.5-flash-8b\", messages=big_history) > MAX_IN:\n    big_history.pop(1)  # drop oldest non-system turn\nresp = completion(model=\"vertex_ai/gemini-1.5-flash-8b\", messages=big_history)","handlingStrategy":"validation","validationCode":"from litellm import token_counter\n\ndef fits_context(model: str, messages: list, headroom: int = 512) -> bool:\n    used = token_counter(model=model, messages=messages)\n    max_in = litellm.get_max_input_tokens(model) if hasattr(litellm, 'get_max_input_tokens') else 1_000_000\n    return used + headroom <= max_in\n\nwhile not fits_context(\"vertex_ai/gemini-1.5-pro\", msgs):\n    msgs.pop(1)  # drop oldest non-system turn","typeGuard":null,"tryCatchPattern":"import litellm\n\ntry:\n    resp = litellm.completion(model=\"vertex_ai/gemini-1.5-pro\", messages=msgs)\nexcept litellm.ContextWindowExceededError:\n    resp = litellm.completion(model=\"vertex_ai/gemini-1.5-pro-002\", messages=msgs)  # 2M fallback","preventionTips":["Token-count before send and truncate/summarize proactively","Use Router context_window_fallbacks to auto-switch to a larger-context model","Track token usage per conversation and warn at 80% of the limit"],"tags":["litellm","vertex-ai","context-window","tokens","prompt-length"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}