{"record":{"id":"b531b1ebd3126cf8","repo":"BerriAI/litellm","slug":"response-text-b531b1","errorCode":null,"errorMessage":"response.text","messagePattern":"response\\.text","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"litellm/llms/together_ai/rerank/handler.py","lineNumber":62,"sourceCode":"        request_data_dict: Final = request_data.dict(exclude_none=True)\n        if max_chunks_per_doc is not None:\n            raise ValueError(\"TogetherAI does not support max_chunks_per_doc\")\n\n        if _is_async:\n            return self.async_rerank(request_data_dict, api_key)  # Call async method\n\n        response: Final = client.post(\n            \"https://api.together.xyz/v1/rerank\",\n            headers={\n                \"accept\": \"application/json\",\n                \"content-type\": \"application/json\",\n                \"authorization\": f\"Bearer {api_key}\",\n            },\n            json=request_data_dict,\n        )\n\n        if response.status_code != 200:\n            raise Exception(response.text)\n\n        _json_response: Final = response.json()\n\n        return TogetherAIRerankConfig()._transform_response(_json_response)\n\n    async def async_rerank(  # New async method\n        self,\n        request_data_dict: dict[str, Any],\n        api_key: str,\n    ) -> RerankResponse:\n        client: Final = get_async_httpx_client(llm_provider=litellm.LlmProviders.TOGETHER_AI)  # Use async client\n\n        response: Final = await client.post(\n            \"https://api.together.xyz/v1/rerank\",\n            headers={\n                \"accept\": \"application/json\",\n                \"content-type\": \"application/json\",\n                \"authorization\": f\"Bearer {api_key}\",","sourceCodeStart":44,"sourceCodeEnd":80,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/together_ai/rerank/handler.py#L44-L80","documentation":"This is the synchronous TogetherAI rerank path: after POSTing to https://api.together.xyz/v1/rerank, any status other than 200 raises a bare Exception whose message is the raw response body (response.text). The underlying causes are server-side rejections — invalid/expired API key (401), unknown model (404/400), malformed query (422), rate limits or Together outages (429/5xx) — surfaced verbatim from Together's error JSON/HTML.","triggerScenarios":"litellm.rerank(model=\"together_ai/rerank-english-v2.0\", ...) with a wrong or revoked Together API key; referencing a retired rerank model id; exceeding rate limits; Together API returning 500 during an incident. Any non-200 produces this exception with the provider's body as the message.","commonSituations":"Key rotation that invalidated a stored TOGETHERAI_API_KEY; deprecation of older rerank model versions; bursty retrival workloads hitting 429s; transient Together 502/503s during deploys.","solutions":["Read the exception message — it is Together's raw error body, which names the actual cause (auth, model, rate limit).","For 401/403: verify the TOGETHERAI_API_KEY used by the call is valid and has rerank access.","For 400/404: confirm the model id exists on Together's models page and is a rerank model (e.g. rerank-english-v2.0).","For 429/5xx: retry with exponential backoff (litellm retries / Router num_retries) and reduce request rate."],"exampleFix":"# before\nresp = litellm.rerank(model=\"together_ai/rerank-english-v2.0\", query=q, documents=docs)\n# Exception: {\"message\":\"Invalid API key\",\"type\":\"invalid_request_error\"}\n\n# after\nfrom litellm import Router\nrouter = Router(\n    model_list=[{\n        \"model_name\": \"together-rerank\",\n        \"litellm_params\": {\"model\": \"together_ai/rerank-english-v2.0\"},\n    }],\n    num_retries=3,           # retries 429/5xx\n    retry_after=2,\n)\nresp = router.rerank(model=\"together-rerank\", query=q, documents=docs)","handlingStrategy":"retry","validationCode":"def together_rerank_request_valid(model: str, api_key: str | None, documents: list) -> tuple[bool, str]:\n    if not api_key:\n        return False, \"missing Together API key\"\n    if not documents:\n        return False, \"documents must be non-empty\"\n    if not model.startswith(\"together_ai/\"):\n        return False, \"model must be a together_ai rerank model\"\n    return True, \"\"","typeGuard":null,"tryCatchPattern":"import time\n\nfor attempt in range(4):\n    try:\n        resp = litellm.rerank(model=\"together_ai/rerank-english-v2.0\", query=q, documents=docs)\n        break\n    except Exception as e:\n        msg = str(e)\n        if \"Invalid API key\" in msg or \"invalid_api_key\" in msg:\n            raise RuntimeError(\"fix TOGETHERAI_API_KEY\") from e  # not retryable\n        if attempt == 3:\n            raise\n        time.sleep(2 ** attempt)  # retry 429/5xx with backoff","preventionTips":["Treat the exception body as Together's error JSON and branch on it: auth → fix key, 4xx model → fix model, 5xx/429 → retry.","Configure Router(num_retries=..., retry_after=...) so backoff is built in.","Log model + document count with every rerank failure for provider triage."],"tags":["together-ai","rerank","http-error","api-response","litellm"],"backgroundTag":"http-error-response","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}