{"record":{"id":"a83f7cb9702aaee9","repo":"run-llama/llama_index","slug":"response-generation-timed-out-after-timeout-seco","errorCode":null,"errorMessage":"Response generation timed out after {timeout} seconds","messagePattern":"Response generation timed out after (.+?) seconds","errorType":"exception","errorClass":"TimeoutError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/langchain_helpers/streaming.py","lineNumber":51,"sourceCode":"        parent_run_id: Optional[UUID] = None,\n        tags: Optional[List[str]] = None,\n        **kwargs: Any,\n    ) -> None:\n        self._done.set()\n\n    def get_response_gen(self, timeout: float = 120.0) -> Generator:\n        \"\"\"\n        Get response generator with timeout.\n\n        Args:\n            timeout (float): Maximum time in seconds to wait for the complete response.\n                            Defaults to 120 seconds.\n\n        \"\"\"\n        start_time = time.time()\n        while True:\n            if time.time() - start_time > timeout:\n                raise TimeoutError(\n                    f\"Response generation timed out after {timeout} seconds\"\n                )\n\n            if not self._token_queue.empty():\n                token = self._token_queue.get_nowait()\n                yield token\n            elif self._done.is_set():\n                break\n            else:\n                # Small sleep to prevent CPU spinning\n                time.sleep(0.01)\n","sourceCodeStart":33,"sourceCodeEnd":63,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/langchain_helpers/streaming.py#L33-L63","documentation":"TimeoutError raised by LangchainChunkBuffer.get_response_gen() in the langchain streaming helpers: the generator polls a token queue every 10ms and, if the complete response has not finished (queue empty and the producer's done-event unset) within `timeout` seconds (default 120), it aborts. It indicates the underlying LLM/streaming callback stalled or is simply slower than the configured budget.","triggerScenarios":"Using the langchain-compatible streaming wrapper around a llama-index query engine and iterating get_response_gen() while the model generates for longer than the timeout; a hung network connection or a callback that never sets the done event; long completions with default timeout=120.","commonSituations":"Wrapping llama-index query engines in LangChain agents with slow local models; reasoning-heavy prompts on large contexts exceeding 2 minutes; a dropped connection that leaves the producer thread blocked so the done flag is never set.","solutions":["Raise the budget: get_response_gen(timeout=600.0) sized to your model's worst-case latency.","Check the underlying stream/connection — if tokens also stopped arriving, the producer is hung and retrying the request is the real fix.","Reduce generation length (max_tokens, tighter prompts) so completion fits the budget.","Catch TimeoutError and re-issue the request or degrade gracefully."],"exampleFix":"# before\nbuffer = LangchainChunkBuffer(...)\nfor chunk in buffer.get_response_gen():  # default 120s -> TimeoutError on long generations\n    ...\n\n# after\nfor chunk in buffer.get_response_gen(timeout=600.0):\n    ...","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    for chunk in buffer.get_response_gen(timeout=600.0):\n        yield chunk\nexcept TimeoutError:\n    # underlying stream stalled: re-issue the request\n    yield from retry_request()","preventionTips":["Size timeout to worst-case model latency, not average","Monitor whether tokens are still flowing to distinguish slow from hung","Set max_tokens so generations fit the budget","Catch TimeoutError separately from other exceptions and retry once"],"tags":["streaming","timeout","langchain","llm"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}