{"record":{"id":"24355bd3065ad954","repo":"infiniflow/ragflow","slug":"operation-timed-out-after-seconds-seconds-and-a","errorCode":null,"errorMessage":"Operation timed out after {seconds} seconds and {attempts} attempts.","messagePattern":"Operation timed out after (.+?) seconds and (.+?) attempts\\.","errorType":"exception","errorClass":"TimeoutError","httpStatus":null,"severity":"error","filePath":"common/connection_utils.py","lineNumber":86,"sourceCode":"            for a in range(attempts):\n                try:\n                    if os.environ.get(\"ENABLE_TIMEOUT_ASSERTION\"):\n                        return await asyncio.wait_for(func(*args, **kwargs), timeout=seconds)\n                    else:\n                        return await func(*args, **kwargs)\n                except asyncio.TimeoutError:\n                    if a < attempts - 1:\n                        continue\n                    if on_timeout is not None:\n                        if callable(on_timeout):\n                            result = on_timeout()\n                            if isinstance(result, Coroutine):\n                                return await result\n                            return result\n                        return on_timeout\n\n                    if exception is None:\n                        raise TimeoutError(f\"Operation timed out after {seconds} seconds and {attempts} attempts.\")\n\n                    if isinstance(exception, BaseException):\n                        raise exception\n\n                    if isinstance(exception, type) and issubclass(exception, BaseException):\n                        raise exception(f\"Operation timed out after {seconds} seconds and {attempts} attempts.\")\n\n                    raise RuntimeError(\"Invalid exception type provided\")\n\n        if asyncio.iscoroutinefunction(func):\n            return async_wrapper\n        return wrapper\n\n    return decorator\n\n\nasync def construct_response(code=RetCode.SUCCESS, message=\"success\", data=None, auth=None):\n    result_dict = {\"code\": code, \"message\": message, \"data\": data}","sourceCodeStart":68,"sourceCodeEnd":104,"githubUrl":"https://github.com/infiniflow/ragflow/blob/554fb1133ac3861732235ad9c377eb5e0a770665/common/connection_utils.py#L68-L104","documentation":"TimeoutError raised by the async branch of the @timeout decorator in common/connection_utils.py:64-94. For coroutine functions, each of `attempts` iterations runs the call under asyncio.wait_for(seconds) (only when ENABLE_TIMEOUT_ASSERTION is set); after the final attempt times out and neither on_timeout nor exception overrides were supplied, the default TimeoutError is raised. Callers can instead supply on_timeout (fallback value/callback) or exception (an exception class/instance to raise instead), which take precedence over this default.","triggerScenarios":"Awaiting an async function decorated with @timeout(seconds=N, attempts=M, exception=None, on_timeout=None) with ENABLE_TIMEOUT_ASSERTION set, where every asyncio.wait_for attempt exceeds N seconds — e.g. an async LLM/chat-completion call hanging past its budget on all retries.","commonSituations":"LLM provider slow or stalled (streaming connection open but no tokens); Elasticsearch/Infinity query exceeding the configured budget under load; test environments asserting timeouts; defaults too tight for reasoning-model requests.","solutions":["Increase seconds (e.g. via the relevant config for LLM timeout) or attempts so the operation fits the budget.","If a degraded-but-working behavior is acceptable, pass on_timeout=... to the decorator instead of letting the default TimeoutError propagate.","Investigate the downstream dependency (LLM endpoint latency, ES query cost) — the decorator only reports the overrun.","Pass exception=SomeSpecificError if callers need to distinguish this timeout from generic TimeoutErrors."],"exampleFix":"# before\n@timeout(seconds=10)\nasync def chat(...): ...\n# after\n@timeout(seconds=120, attempts=2, exception=LLMTimeoutError)\nasync def chat(...): ...","handlingStrategy":"fallback","validationCode":"if os.getenv(\"ENABLE_TIMEOUT_ASSERTION\") and expected_p95_latency > configured_seconds:\n    raise RuntimeError(f\"timeout budget {configured_seconds}s < p95 latency {expected_p95_latency}s — raise @timeout seconds\")","typeGuard":null,"tryCatchPattern":"try:\n    out = await decorated_call()\nexcept TimeoutError as e:\n    if \"Operation timed out\" in str(e):\n        return cached_or_default_result  # fallback path\n    raise","preventionTips":["Pass on_timeout=... to the decorator when a degraded default beats raising.","Budget seconds above the downstream p99 latency, especially for LLM calls.","Use exception=CustomTimeoutError to make catches precise."],"tags":["timeout","asyncio","decorator","llm","network"],"backgroundTag":null,"analyzedSha":"554fb1133ac3861732235ad9c377eb5e0a770665","analyzedAt":"2026-08-15T09:20:16.380Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}