{"record":{"id":"6cb6d253d7c5d127","repo":"calesthio/OpenMontage","slug":"could-not-find-download-url-on-esa-detail-page-d","errorCode":null,"errorMessage":"Could not find download URL on ESA detail page: {detail_url}","messagePattern":"Could not find download URL on ESA detail page: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"tools/video/stock_sources/esa.py","lineNumber":181,"sourceCode":"                href = a.get(\"href\", \"\")\n                text = a.get_text(strip=True).lower()\n                if any(ext in href.lower() for ext in [\".mp4\", \".mov\", \".webm\"]):\n                    download_url = href\n                    break\n                if \"download\" in text and href:\n                    download_url = href\n                    break\n\n            # Check for video source tags\n            if not download_url:\n                for source in soup.select(\"video source[src], source[src]\"):\n                    src = source.get(\"src\", \"\")\n                    if src:\n                        download_url = src\n                        break\n\n            if not download_url:\n                raise ValueError(f\"Could not find download URL on ESA detail page: {detail_url}\")\n\n            if not download_url.startswith(\"http\"):\n                download_url = f\"https://www.esa.int{download_url}\"\n\n            return self._stream_download(download_url, out_path)\n\n        except Exception as e:\n            raise RuntimeError(f\"ESA download failed for {detail_url}: {e}\") from e\n\n    def _stream_download(self, url: str, out_path: Path) -> Path:\n        import requests\n\n        with requests.get(\n            url, stream=True, timeout=180,\n            headers={\"User-Agent\": \"OpenMontage/1.0\"},\n        ) as r:\n            r.raise_for_status()\n            with open(out_path, \"wb\") as f:","sourceCodeStart":163,"sourceCodeEnd":199,"githubUrl":"https://github.com/calesthio/OpenMontage/blob/95e1c3d0ab93482159818560f6a8c8e866b9139f/tools/video/stock_sources/esa.py#L163-L199","documentation":"ValueError raised by the ESA adapter's download() when the esa.int detail page yields neither a matching anchor href nor a <video><source src> / <source src> element. Like Dareful, ESA is scraped from HTML; the error means the expected download markup was absent — page template change, media served only via a player/CDN blob, or the asset moved.","triggerScenarios":"ESA.int redesigns their multimedia pages; the video is embedded via a JavaScript player with no static <source> tag; the detail_url points to a gallery index rather than a single asset; a cookie/consent wall returns different HTML.","commonSituations":"Public-agency sites (ESA) revamping CMS with no API stability guarantee; EU cookie-consent interstitials altering the served DOM; old cached URLs from previously built corpora.","solutions":["Verify the detail_url still resolves to a single-asset page with a visible video element.","Extract the direct media URL from the page's network tab in a browser and download it directly if the adapter's selectors miss.","Update the CSS selectors in esa.py if the markup merely changed shape but media elements remain.","Fall back to another stock source for that clip."],"exampleFix":"# before\npath = esa.download(candidate, out)\n\n# after\ntry:\n    path = esa.download(candidate, out)\nexcept (ValueError, RuntimeError):\n    path = get_source(\"archive_org\").download(alternate_candidate, out)","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    path = esa.download(candidate, out_path)\nexcept (ValueError, RuntimeError) as e:\n    logger.warning(\"esa scrape failed: %s\", e)\n    path = get_source(\"archive_org\").download(alt_candidate, out_path)","preventionTips":["Expect agency-site scrapers to break on redesigns; isolate them behind a source interface with alternates.","Capture direct media URLs when available and persist them for resumable downloads."],"tags":["stock-footage","esa","scraping","fragile","download"],"backgroundTag":null,"analyzedSha":"95e1c3d0ab93482159818560f6a8c8e866b9139f","analyzedAt":"2026-08-15T06:31:20.014Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}