{"record":{"id":"cfb995e10efd88ed","repo":"BerriAI/litellm","slug":"togetheraiexception-error-response-error","errorCode":null,"errorMessage":"TogetherAIException - {error_response['error']}","messagePattern":"TogetherAIException - (.+?)","errorType":"exception","errorClass":"ContextWindowExceededError","httpStatus":400,"severity":"error","filePath":"litellm/litellm_core_utils/exception_mapping_utils.py","lineNumber":1662,"sourceCode":"            )\n\n\ndef _map_together_ai_exception(\n    *,\n    model: str,\n    original_exception: _ProviderHTTPException,\n    custom_llm_provider: str,\n    error_str: str,\n    exception_type: str,\n    exception_provider: str,\n    extra_information: str,\n) -> None:\n    try:\n        error_response = json.loads(error_str)\n    except Exception:\n        error_response = {\"error\": error_str}\n    if \"error\" in error_response and \"`inputs` tokens + `max_new_tokens` must be <=\" in error_response[\"error\"]:\n        raise ContextWindowExceededError(\n            message=f\"TogetherAIException - {error_response['error']}\",\n            model=model,\n            llm_provider=\"together_ai\",\n            response=getattr(original_exception, \"response\", None),\n        )\n    elif \"error\" in error_response and \"invalid private key\" in error_response[\"error\"]:\n        raise AuthenticationError(\n            message=f\"TogetherAIException - {error_response['error']}\",\n            llm_provider=\"together_ai\",\n            model=model,\n            response=getattr(original_exception, \"response\", None),\n        )\n    elif \"error\" in error_response and \"INVALID_ARGUMENT\" in error_response[\"error\"]:\n        raise BadRequestError(\n            message=f\"TogetherAIException - {error_response['error']}\",\n            model=model,\n            llm_provider=\"together_ai\",\n            response=getattr(original_exception, \"response\", None),","sourceCodeStart":1644,"sourceCodeEnd":1680,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/litellm_core_utils/exception_mapping_utils.py#L1644-L1680","documentation":"This is a litellm ContextWindowExceededError raised by the Together AI exception mapper. The mapper first json.loads the error body; if it contains an 'error' key whose text includes '`inputs` tokens + `max_new_tokens` must be <=', Together AI has rejected the request because prompt tokens plus requested generation length exceed the model's context window. litellm normalizes this to its context-window class.","triggerScenarios":"Calling a together_ai model with (prompt tokens + max_tokens) above the model context limit — e.g. long prompt with max_tokens=2048 on a 4096-context model; the check is on the SUM, not just the prompt.","commonSituations":"Developers setting generous max_tokens 'to be safe' which then overflows the budget; long RAG contexts; switching to a smaller-context model without adjusting max_tokens.","solutions":["Reduce max_tokens so prompt_tokens + max_tokens <= model context window","Truncate/summarize the prompt","Look up the model's limit: litellm.get_max_tokens(model)","Or switch to a larger-context Together model"],"exampleFix":"# before\nlitellm.completion(model=\"together_ai/Together-Code-7B\", messages=msgs, max_tokens=2048)\n\n# after\nctx = litellm.get_max_tokens(\"together_ai/Together-Code-7B\")\nprompt_toks = len(litellm.encode(model=\"together_ai/Together-Code-7B\", text=msgs_text))\nlitellm.completion(..., max_tokens=min(512, ctx - prompt_toks - 8))","handlingStrategy":"validation","validationCode":"ctx = litellm.get_max_tokens(model) or 4096\nprompt_tokens = sum(\n    len(litellm.encode(model=model, text=m[\"content\"])) for m in messages\n)\nif prompt_tokens + max_tokens > ctx:\n    max_tokens = max(64, ctx - prompt_tokens - 8)  # shrink output budget\n    messages = trim_to(messages, ctx - max_tokens - 8)","typeGuard":"def is_context_window_error(e: Exception) -> bool:\n    return isinstance(e, litellm.ContextWindowExceededError)","tryCatchPattern":"try:\n    resp = litellm.completion(model=model, messages=messages, max_tokens=max_tokens)\nexcept litellm.ContextWindowExceededError:\n    resp = litellm.completion(model=model, messages=trim(messages, 0.5), max_tokens=256)","preventionTips":["Budget tokens: prompt_tokens + max_tokens must be < context window","Compute prompt token counts with litellm.encode before large calls","Don't set max_tokens reflexively high; size it to actual need"],"tags":["together-ai","context-window","max-tokens","token-limit","litellm"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}