{"record":{"id":"d1cfe3dfd3ceca6f","repo":"ZhuLinsen/daily_stock_analysis","slug":"task-name-ranking-fetch-in-flight-limit-reached","errorCode":null,"errorMessage":"{task_name} ranking fetch in-flight limit reached","messagePattern":"(.+?) ranking fetch in-flight limit reached","errorType":"exception","errorClass":"TimeoutError","httpStatus":null,"severity":"warning","filePath":"src/services/market_hotspot_service.py","lineNumber":486,"sourceCode":"                        f\"{task_name} ranking fetch cooling down after previous timeout\"\n                    )\n                cls._ranking_fetch_retry_after.pop(inflight_key, None)\n                if cls._ranking_fetch_futures.get(inflight_key) is retry_future:\n                    cls._ranking_fetch_futures.pop(inflight_key, None)\n                    if retry_future.done() or retry_future.cancelled():\n                        cls._ranking_fetch_slots.release()\n                    else:\n                        cls._ranking_fetch_detached_futures.add(retry_future)\n\n            current = cls._ranking_fetch_futures.get(inflight_key)\n            if current is not None:\n                if not current.done():\n                    return current\n                cls._ranking_fetch_futures.pop(inflight_key, None)\n                cls._ranking_fetch_slots.release()\n\n            if not cls._ranking_fetch_slots.acquire(blocking=False):\n                raise TimeoutError(f\"{task_name} ranking fetch in-flight limit reached\")\n\n            future: Future = Future()\n            cls._ranking_fetch_retry_after.pop(inflight_key, None)\n            cls._ranking_fetch_futures[inflight_key] = future\n            future.add_done_callback(\n                lambda done_future: cls._forget_ranking_fetch(inflight_key, done_future)\n            )\n            worker = threading.Thread(\n                target=cls._run_ranking_fetch,\n                args=(future, task),\n                daemon=True,\n                name=f\"market-hotspot-{task_name}\",\n            )\n            submitted = future\n        try:\n            worker.start()\n        except BaseException as exc:\n            cls._drop_unstarted_ranking_fetch(inflight_key, submitted)","sourceCodeStart":468,"sourceCodeEnd":504,"githubUrl":"https://github.com/ZhuLinsen/daily_stock_analysis/blob/5159bd72e8373d215492dff122acc9d389e219c9/src/services/market_hotspot_service.py#L468-L504","documentation":"TimeoutError(f'{task_name} ranking fetch in-flight limit reached') is raised by MarketHotspotService._get_or_submit_ranking_fetch when the non-blocking acquire of _ranking_fetch_slots fails — the global cap on concurrent ranking fetch worker threads is exhausted. The semaphore is only released when futures complete or are forgotten, so the cap reflects genuinely busy workers plus any not-yet-reaped finished ones.","triggerScenarios":"Issuing more distinct ranking fetches concurrently than the configured slot count (each unique inflight_key holds a slot until done), e.g. fanning out all markets/boards at once; or slots leaked by detached futures that never complete.","commonSituations":"A burst scheduler requesting all rankings simultaneously at market open, retry storms after partial timeouts holding slots, or worker threads stuck on a hanging upstream connection never releasing their slot.","solutions":["Reduce fan-out: stagger or batch ranking fetches so concurrent distinct fetches stay under the slot limit","Check for stuck workers (thread named 'market-hotspot-<task>') and hanging sockets; ensure upstream requests have their own socket timeout","Retry shortly after — completed futures release slots via _forget_ranking_fetch, so the limit is transient","If the workload legitimately needs more parallelism, raise the ranking fetch slot configuration rather than bypassing the semaphore"],"exampleFix":"# before: all markets fetched at once exhausts slots\nresults = {m: fetch_ranking(m) for m in ALL_MARKETS}  # raises for some\n\n# after: bounded concurrency\nwith ThreadPoolExecutor(max_workers=2) as pool:\n    results = dict(zip(ALL_MARKETS, pool.map(fetch_ranking, ALL_MARKETS)))","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    data = fetch_ranking(key)\nexcept TimeoutError as exc:\n    if \"in-flight limit reached\" in str(exc):\n        time.sleep(0.1)  # slots free as futures complete\n        data = fetch_ranking(key)","preventionTips":["Bound concurrent distinct ranking fetches below the slot limit","Stagger bulk fetch jobs (e.g. at market open) instead of fanning out all at once","Monitor for stuck 'market-hotspot-*' threads that leak slots"],"tags":["concurrency","rate-limit","market-data","timeout"],"backgroundTag":null,"analyzedSha":"5159bd72e8373d215492dff122acc9d389e219c9","analyzedAt":"2026-08-15T01:59:36.292Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}