{"record":{"id":"277a6fcd4e302b3c","repo":"666ghj/MiroFish","slug":"zep-simulation-ingestion-timed-out-with-len-pendi","errorCode":null,"errorMessage":"Zep simulation ingestion timed out with {len(pending)} episode(s) pending","messagePattern":"Zep simulation ingestion timed out with (.+?) episode\\(s\\) pending","errorType":"exception","errorClass":"TimeoutError","httpStatus":null,"severity":"error","filePath":"backend/app/services/zep_graph_memory_updater.py","lineNumber":608,"sourceCode":"                )\n            except _DrainDeadlineExceeded as error:\n                with self._buffer_lock:\n                    del self._platform_buffers[platform][:error.processed_count]\n                raise TimeoutError(str(error)) from error\n            else:\n                with self._buffer_lock:\n                    del self._platform_buffers[platform][:processed_count]\n\n    def _wait_for_pending_episodes(self, *, deadline: float | None = None) -> None:\n        pending = set(self._pending_episode_uuids)\n        if not pending:\n            return\n\n        if deadline is None:\n            deadline = time.time() + ZEP_INGESTION_WAIT_TIMEOUT_SECONDS\n        while pending:\n            if time.time() >= deadline:\n                raise TimeoutError(\n                    f\"Zep simulation ingestion timed out with {len(pending)} \"\n                    \"episode(s) pending\"\n                )\n            for episode_uuid in list(pending):\n                episode = call_zep_read_with_retry(\n                    lambda: self.client.graph.episode.get(uuid_=episode_uuid),\n                    operation_name=f\"poll simulation episode {episode_uuid}\",\n                )\n                if getattr(episode, \"processed\", False):\n                    pending.remove(episode_uuid)\n            if pending:\n                time.sleep(3)\n        self._pending_episode_uuids = []\n    \n    def get_stats(self) -> Dict[str, Any]:\n        \"\"\"获取统计信息\"\"\"\n        with self._buffer_lock:\n            buffer_sizes = {p: len(b) for p, b in self._platform_buffers.items()}","sourceCodeStart":590,"sourceCodeEnd":626,"githubUrl":"https://github.com/666ghj/MiroFish/blob/b5b53acc57189a4a42e44a23e149dc655c98fe82/backend/app/services/zep_graph_memory_updater.py#L590-L626","documentation":"Raised by _wait_for_pending_episodes when polling Zep's asynchronous ingestion: episodes accepted by graph.add are polled via client.graph.episode.get(uuid_=...) until episode.processed is True, and if the deadline (default ZEP_INGESTION_WAIT_TIMEOUT_SECONDS = 600s, polled every 3s) passes with episodes still unprocessed, this TimeoutError fires with the pending count.","triggerScenarios":"After sending episodes, polling loop hits time.time() >= deadline while at least one episode still has processed=False. Triggered by heavy Zep Cloud ingestion queues, large batches of extracted facts, or Zep-side outages that leave episodes perpetually unprocessed.","commonSituations":"Simulations that push hundreds of activities near the end of the ingestion window; Zep Cloud load or maintenance slowing extraction; episodes rejected server-side without being marked processed. The 600s constant may be too short for very large knowledge-graph updates.","solutions":["Retry the wait with a fresh deadline — episodes may still complete; re-poll the same UUIDs before assuming failure","Increase ZEP_INGESTION_WAIT_TIMEOUT_SECONDS for large simulations","Check Zep Cloud status/queue latency; reduce batch sizes so extraction finishes faster","If an episode never becomes processed, fetch it via graph.episode.get and inspect its state for server-side errors","Send activities incrementally during the simulation instead of a large end-of-run flush"],"exampleFix":"# before\nZEP_INGESTION_WAIT_TIMEOUT_SECONDS = 600\n\n# after (sized to worst observed ingestion latency)\nZEP_INGESTION_WAIT_TIMEOUT_SECONDS = 900","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    updater._wait_for_pending_episodes(deadline=d)\nexcept TimeoutError:\n    # episodes may still complete later; re-poll before declaring failure\n    updater._wait_for_pending_episodes(deadline=time.time() + EXTRA_WAIT)","preventionTips":["Increase ZEP_INGESTION_WAIT_TIMEOUT_SECONDS for large simulations","Flush activities throughout the run so ingestion overlaps with simulation time","Track per-episode UUIDs so a retry polls only genuinely pending episodes"],"tags":["zep","ingestion","timeout","polling"],"backgroundTag":null,"analyzedSha":"b5b53acc57189a4a42e44a23e149dc655c98fe82","analyzedAt":"2026-08-14T22:29:33.146Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}