BerriAI/litellm · error · BedrockError

Timeout error occurred.

Error message

Timeout error occurred.

What it means

Async rerank timeout guard: the awaited POST to the Bedrock agent-runtime /rerank endpoint exceeded the httpx timeout; litellm maps httpx.TimeoutException to BedrockError(408, 'Timeout error occurred.'). Rerank latency grows with document count, so large batches frequently trip short timeouts.

Source

Thrown at litellm/llms/bedrock/rerank/handler.py:49

        prepared_request: BedrockPreparedRequest,
        timeout: float | httpx.Timeout | None = None,
        client: AsyncHTTPHandler | None = None,
    ):
        if client is None:
            client = get_async_httpx_client(llm_provider=litellm.LlmProviders.BEDROCK)
        try:
            response: Final = await client.post(
                url=prepared_request["endpoint_url"],
                headers=dict(prepared_request["prepped"].headers),
                data=prepared_request["body"],
                timeout=timeout,
            )
            response.raise_for_status()
        except httpx.HTTPStatusError as err:
            error_code: Final = err.response.status_code
            raise BedrockError(status_code=error_code, message=err.response.text)
        except httpx.TimeoutException:
            raise BedrockError(status_code=408, message="Timeout error occurred.")

        return BedrockRerankConfig()._transform_response(response.json())

    def rerank(
        self,
        model: str,
        query: str,
        documents: list[str | dict[str, Any]],
        optional_params: dict,
        logging_obj: LitellmLogging,
        top_n: int | None = None,
        rank_fields: list[str] | None = None,
        return_documents: bool | None = True,
        max_chunks_per_doc: int | None = None,
        _is_async: bool | None = False,
        timeout: float | httpx.Timeout | None = None,
        api_base: str | None = None,
        extra_headers: dict | None = None,

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Pass a larger timeout: litellm.rerank(..., timeout=120).
2. Batch documents into smaller rerank calls and merge scores client-side.
3. Retry on the 408 BedrockError with backoff.
4. Pre-truncate document text to the needed context length before sending.

Example fix

# before
result = await litellm.rerank(model=MODEL, query=q, documents=docs, timeout=10)

# after
result = await litellm.rerank(model=MODEL, query=q, documents=docs, timeout=120)

def estimate_rerank_timeout(n_docs: int, avg_chars: int) -> float:
    # heuristic: scale timeout with corpus size
    return min(300, 10 + n_docs * (0.05 + avg_chars / 20000))

try:
    result = await litellm.rerank(model=model, query=q, documents=docs, timeout=120)
except BedrockError as e:
    if e.status_code == 408:
        # split and retry smaller batches instead of one long call
        results = await asyncio.gather(*[
            litellm.rerank(model=model, query=q, documents=batch, timeout=60)
            for batch in chunk(docs, 25)
        ])
    else:
        raise

When it happens

Trigger: await litellm.rerank(model='bedrock/amazon.rerank-v1:0', documents=[...hundreds...], timeout=N) where N is too small for the corpus size; also default timeout used on slow networks.

Common situations: Reranking 100+ long documents in one call; congested network; region far from client; concurrent bursts triggering queueing on the endpoint.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• err.response.text
• BedrockException: Timeout Error - {error_str}
• output_s3_uri is required for async invoke requests
• Timeout error occurred.
• Timeout error occurred.

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/7f6700d6432fa89d. Report an issue: GitHub.