BerriAI/litellm · error · Timeout

APITimeoutError - Request timed out. Error_str

Error message

APITimeoutError - Request timed out. Error_str: {error_str}

What it means

Inside exception_type(), before provider-specific mapping, litellm string-matches the error text for 'Request Timeout Error', 'Request timed out', 'Timed out generating response', or 'The read operation timed out' and raises litellm.Timeout ('APITimeoutError - Request timed out. Error_str: ...'). This catches client/gateway timeout exceptions (openai.APITimeoutError, httpx read timeouts, vertex read timeouts) that surface as text rather than status codes.

Solutions

Increase the timeout: litellm.completion(..., timeout=600) or litellm.timeout = 600 globally.
Use streaming so partial output arrives within the read window.
Reduce per-request work (max_tokens, input size) or parallelize smaller chunks instead of one giant call.
Enable num_retries so transient timeouts are retried automatically.

Example fix

# before
litellm.completion(model="gpt-4o", messages=big_msgs, timeout=30)

# after
litellm.completion(
    model="gpt-4o",
    messages=big_msgs,
    timeout=600,
    num_retries=3,
)

Defensive patterns

Strategy: retry

Validate before calling

# budget check: estimated generation time vs configured timeout
est_tokens = litellm.token_counter(model=model, messages=msgs)
assert timeout >= est_tokens / TOKENS_PER_SEC.get(model, 50), "timeout smaller than expected generation time"

Try / catch

try:
    resp = litellm.completion(**kwargs)
except litellm.Timeout as e:
    if "APITimeoutError" in str(e):
        kwargs["timeout"] = (kwargs.get("timeout") or 60) * 2
        kwargs["num_retries"] = 2
        resp = litellm.completion(**kwargs)
    else:
        raise

Prevention

Set litellm.timeout (or per-call timeout=) explicitly — never rely on defaults for long jobs.
Stream long generations; non-streaming waits pay the full latency in one window.
Enable num_retries for automatic timeout retries.
Token-count inputs to predict slow calls before making them.

When it happens

Trigger: The HTTP client's own timeout elapsing before any provider response: openai-python APITimeoutError, httpx.ReadTimeout/WriteTimeout wrapped by a provider adapter, or provider SDKs whose timeout messages contain those phrases. Also triggered by user-supplied timeout= being smaller than generation time.

Common situations: Default 600s/short timeouts with very long generations; slow embeddings on large inputs; high-latency self-hosted/proxied endpoints; retrying heavy batch jobs with the same tiny timeout.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/2b67a513c482dc40. Report an issue: GitHub.

Appendix: source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:2256

                pass

            ################################################################################
            # End of Common Extra information Needed for all providers
            ################################################################################

            ################################################################################
            #################### Start of Provider Exception mapping ####################
            ################################################################################

            if (
                "Request Timeout Error" in error_str
                or "Request timed out" in error_str
                or "Timed out generating response" in error_str
                or "The read operation timed out" in error_str
            ):
                exception_mapping_worked = True

                raise Timeout(
                    message=f"APITimeoutError - Request timed out. Error_str: {error_str}",
                    model=model,
                    llm_provider=custom_llm_provider,
                    litellm_debug_info=extra_information,
                )

            if (
                custom_llm_provider == "litellm_proxy"
            ):  # handle special case where calling litellm proxy + exception str contains error message
                extract_and_raise_litellm_exception(
                    response=getattr(original_exception, "response", None),
                    error_str=error_str,
                    model=model,
                    custom_llm_provider=custom_llm_provider,
                )
            if (
                custom_llm_provider == "openai"
                or custom_llm_provider == "text-completion-openai"

View on GitHub (pinned to 77b7c6c40c)