{"record":{"id":"d5ca49ce0c7725c6","repo":"BerriAI/litellm","slug":"togetheraiexception-original-exception-message","errorCode":null,"errorMessage":"TogetherAIException - {original_exception.message}","messagePattern":"TogetherAIException - (.+?)","errorType":"exception","errorClass":"Timeout","httpStatus":408,"severity":"error","filePath":"litellm/litellm_core_utils/exception_mapping_utils.py","lineNumber":1702,"sourceCode":"            message=f\"TogetherAIException - {error_str}\",\n            model=model,\n            llm_provider=\"together_ai\",\n        )\n    elif (\n        \"error\" in error_response\n        and \"API key doesn't match expected format.\" in error_response[\"error\"]\n        or \"error_type\" in error_response\n        and error_response[\"error_type\"] == \"validation\"\n    ):\n        raise BadRequestError(\n            message=f\"TogetherAIException - {error_response['error']}\",\n            model=model,\n            llm_provider=\"together_ai\",\n            response=getattr(original_exception, \"response\", None),\n        )\n    if hasattr(original_exception, \"status_code\"):\n        if original_exception.status_code == 408:\n            raise Timeout(\n                message=f\"TogetherAIException - {original_exception.message}\",\n                model=model,\n                llm_provider=\"together_ai\",\n            )\n        elif original_exception.status_code == 422:\n            raise BadRequestError(\n                message=f\"TogetherAIException - {error_response['error']}\",\n                model=model,\n                llm_provider=\"together_ai\",\n                response=getattr(original_exception, \"response\", None),\n            )\n        elif original_exception.status_code == 429:\n            raise RateLimitError(\n                message=f\"TogetherAIException - {original_exception.message}\",\n                llm_provider=\"together_ai\",\n                model=model,\n                response=getattr(original_exception, \"response\", None),\n            )","sourceCodeStart":1684,"sourceCodeEnd":1720,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/litellm_core_utils/exception_mapping_utils.py#L1684-L1720","documentation":"litellm translates an HTTP 408 from the Together AI API into litellm.Timeout ('TogetherAIException - ...'). It means Together's server accepted the request but it did not complete within its gateway timeout window. litellm re-raises it as a Timeout so callers can apply uniform retry logic.","triggerScenarios":"A together_ai completion/embedding call whose server-side processing exceeds Together's request timeout, so the API returns status 408. Typical with very large prompts, huge max_tokens, or slow non-streaming generations.","commonSituations":"Long-context jobs on Mixtral/Llama models routed via model='together_ai/...'; sudden latency spikes on Together's side; requests serialized through slow middlewares that inflate processing time.","solutions":["Retry with exponential backoff (408 is transient) using litellm.Timeout as the catch signal","Reduce max_tokens, trim the prompt, or set stream=True so tokens flow before the gateway timeout","Pass an explicit timeout to litellm.completion(...) so the client gives up before the server does","If persistent, check Together AI status/usage dashboard for degraded throughput"],"exampleFix":"// before\nresp = litellm.completion(model='together_ai/mistralai/Mixtral-8x7B-Instruct-v0.1', messages=msgs, max_tokens=4096)\n// after\nresp = litellm.completion(model='together_ai/mistralai/Mixtral-8x7B-Instruct-v0.1', messages=msgs, max_tokens=1024, stream=True, timeout=120)","handlingStrategy":"retry","validationCode":"prompt_tokens = litellm.token_counter(model='together_ai/mistralai/Mixtral-8x7B-Instruct-v0.1', messages=msgs)\nif prompt_tokens > 20_000:  # long prompts push past gateway timeout\n    raise ValueError('Trim prompt or stream before calling Together AI')","typeGuard":"import litellm\n\ndef is_together_timeout(e: Exception) -> bool:\n    return isinstance(e, litellm.Timeout) and 'TogetherAIException' in str(e)","tryCatchPattern":"for attempt in range(3):\n    try:\n        return litellm.completion(...)\n    except litellm.Timeout:\n        time.sleep(2 ** attempt)\nraise RuntimeError('Together AI timed out after retries')","preventionTips":["Keep max_tokens modest for non-streaming calls","Stream long generations","Wrap together_ai calls in bounded retry with jittered backoff"],"tags":["together-ai","timeout","retry","litellm"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}