{"record":{"id":"fded6d607fc1a360","repo":"calesthio/OpenMontage","slug":"esa-download-failed-for-detail-url-e","errorCode":null,"errorMessage":"ESA download failed for {detail_url}: {e}","messagePattern":"ESA download failed for (.+?): (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"warning","filePath":"tools/video/stock_sources/esa.py","lineNumber":189,"sourceCode":"\n            # Check for video source tags\n            if not download_url:\n                for source in soup.select(\"video source[src], source[src]\"):\n                    src = source.get(\"src\", \"\")\n                    if src:\n                        download_url = src\n                        break\n\n            if not download_url:\n                raise ValueError(f\"Could not find download URL on ESA detail page: {detail_url}\")\n\n            if not download_url.startswith(\"http\"):\n                download_url = f\"https://www.esa.int{download_url}\"\n\n            return self._stream_download(download_url, out_path)\n\n        except Exception as e:\n            raise RuntimeError(f\"ESA download failed for {detail_url}: {e}\") from e\n\n    def _stream_download(self, url: str, out_path: Path) -> Path:\n        import requests\n\n        with requests.get(\n            url, stream=True, timeout=180,\n            headers={\"User-Agent\": \"OpenMontage/1.0\"},\n        ) as r:\n            r.raise_for_status()\n            with open(out_path, \"wb\") as f:\n                for chunk in r.iter_content(chunk_size=1 << 16):\n                    if chunk:\n                        f.write(chunk)\n        return out_path\n","sourceCodeStart":171,"sourceCodeEnd":204,"githubUrl":"https://github.com/calesthio/OpenMontage/blob/95e1c3d0ab93482159818560f6a8c8e866b9139f/tools/video/stock_sources/esa.py#L171-L204","documentation":"RuntimeError raised by the ESA adapter's outer except, wrapping every download-path failure — the detail-page scrape miss (358), HTTP errors from _stream_download, or timeouts (180s). The wrapped cause is preserved via 'from e', and the '{e}' suffix carries the underlying message, making this the unified ESA failure surface.","triggerScenarios":"Expired or moved esa.int asset URLs returning 404, 403 from the ESA CDN, requests.Timeout on slow multi-hundred-MB space footage, or any parse exception while building the soup — all surface as this RuntimeError.","commonSituations":"Long-running corpus builders where ESA links rot over months; large-format mission footage exceeding the 180s stream timeout on slow links; intermittent CDN blocks of non-browser user agents.","solutions":["Check __cause__ to separate scraping failures from HTTP/timeout failures.","Make ESA per-clip failures non-fatal in batch pipelines: catch, log, backfill from Archive.org or Pexels.","For timeouts, download large ESA assets directly with a resumable downloader (curl -C -, wget -c) using the same URL.","Refresh search results rather than reusing old candidates whose URLs may have expired."],"exampleFix":"# before\npath = esa.download(candidate, out)\n\n# after\ntry:\n    path = esa.download(candidate, out)\nexcept RuntimeError as e:\n    logger.warning(\"esa clip %s failed: %s\", candidate.clip_id, e.__cause__ or e)\n    path = backfill_from_alternate_source(candidate, out)","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"for c in candidates:\n    try:\n        paths.append(esa.download(c, out))\n    except RuntimeError as e:\n        logger.warning(\"esa %s failed (%s), continuing\", c.clip_id, e.__cause__ or e)\n        continue","preventionTips":["Use resumable downloads (curl -C -) for large ESA footage instead of one 180s stream.","Rotate sources on failure; never let one scraped site block a batch build."],"tags":["stock-footage","esa","download","error-wrapping"],"backgroundTag":null,"analyzedSha":"95e1c3d0ab93482159818560f6a8c8e866b9139f","analyzedAt":"2026-08-15T06:31:20.014Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}