{"record":{"id":"7f6700d6432fa89d","repo":"BerriAI/litellm","slug":"timeout-error-occurred-7f6700","errorCode":null,"errorMessage":"Timeout error occurred.","messagePattern":"Timeout error occurred\\.","errorType":"exception","errorClass":"BedrockError","httpStatus":408,"severity":"error","filePath":"litellm/llms/bedrock/rerank/handler.py","lineNumber":49,"sourceCode":"        prepared_request: BedrockPreparedRequest,\n        timeout: float | httpx.Timeout | None = None,\n        client: AsyncHTTPHandler | None = None,\n    ):\n        if client is None:\n            client = get_async_httpx_client(llm_provider=litellm.LlmProviders.BEDROCK)\n        try:\n            response: Final = await client.post(\n                url=prepared_request[\"endpoint_url\"],\n                headers=dict(prepared_request[\"prepped\"].headers),\n                data=prepared_request[\"body\"],\n                timeout=timeout,\n            )\n            response.raise_for_status()\n        except httpx.HTTPStatusError as err:\n            error_code: Final = err.response.status_code\n            raise BedrockError(status_code=error_code, message=err.response.text)\n        except httpx.TimeoutException:\n            raise BedrockError(status_code=408, message=\"Timeout error occurred.\")\n\n        return BedrockRerankConfig()._transform_response(response.json())\n\n    def rerank(\n        self,\n        model: str,\n        query: str,\n        documents: list[str | dict[str, Any]],\n        optional_params: dict,\n        logging_obj: LitellmLogging,\n        top_n: int | None = None,\n        rank_fields: list[str] | None = None,\n        return_documents: bool | None = True,\n        max_chunks_per_doc: int | None = None,\n        _is_async: bool | None = False,\n        timeout: float | httpx.Timeout | None = None,\n        api_base: str | None = None,\n        extra_headers: dict | None = None,","sourceCodeStart":31,"sourceCodeEnd":67,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/bedrock/rerank/handler.py#L31-L67","documentation":"Async rerank timeout guard: the awaited POST to the Bedrock agent-runtime /rerank endpoint exceeded the httpx timeout; litellm maps httpx.TimeoutException to BedrockError(408, 'Timeout error occurred.'). Rerank latency grows with document count, so large batches frequently trip short timeouts.","triggerScenarios":"await litellm.rerank(model='bedrock/amazon.rerank-v1:0', documents=[...hundreds...], timeout=N) where N is too small for the corpus size; also default timeout used on slow networks.","commonSituations":"Reranking 100+ long documents in one call; congested network; region far from client; concurrent bursts triggering queueing on the endpoint.","solutions":["Pass a larger timeout: litellm.rerank(..., timeout=120).","Batch documents into smaller rerank calls and merge scores client-side.","Retry on the 408 BedrockError with backoff.","Pre-truncate document text to the needed context length before sending."],"exampleFix":"# before\nresult = await litellm.rerank(model=MODEL, query=q, documents=docs, timeout=10)\n\n# after\nresult = await litellm.rerank(model=MODEL, query=q, documents=docs, timeout=120)","handlingStrategy":"retry","validationCode":"def estimate_rerank_timeout(n_docs: int, avg_chars: int) -> float:\n    # heuristic: scale timeout with corpus size\n    return min(300, 10 + n_docs * (0.05 + avg_chars / 20000))","typeGuard":null,"tryCatchPattern":"try:\n    result = await litellm.rerank(model=model, query=q, documents=docs, timeout=120)\nexcept BedrockError as e:\n    if e.status_code == 408:\n        # split and retry smaller batches instead of one long call\n        results = await asyncio.gather(*[\n            litellm.rerank(model=model, query=q, documents=batch, timeout=60)\n            for batch in chunk(docs, 25)\n        ])\n    else:\n        raise","preventionTips":["Scale the timeout with document count/size.","Chunk large document sets into batches.","Treat 408 from rerank as retryable with backoff."],"tags":["bedrock","rerank","timeout","async"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}