calesthio/OpenMontage · warning · RuntimeError

ESA download failed for {detail_url}: {e}

Error message

ESA download failed for {detail_url}: {e}

What it means

RuntimeError raised by the ESA adapter's outer except, wrapping every download-path failure — the detail-page scrape miss (358), HTTP errors from _stream_download, or timeouts (180s). The wrapped cause is preserved via 'from e', and the '{e}' suffix carries the underlying message, making this the unified ESA failure surface.

Source

Thrown at tools/video/stock_sources/esa.py:189

            # Check for video source tags
            if not download_url:
                for source in soup.select("video source[src], source[src]"):
                    src = source.get("src", "")
                    if src:
                        download_url = src
                        break

            if not download_url:
                raise ValueError(f"Could not find download URL on ESA detail page: {detail_url}")

            if not download_url.startswith("http"):
                download_url = f"https://www.esa.int{download_url}"

            return self._stream_download(download_url, out_path)

        except Exception as e:
            raise RuntimeError(f"ESA download failed for {detail_url}: {e}") from e

    def _stream_download(self, url: str, out_path: Path) -> Path:
        import requests

        with requests.get(
            url, stream=True, timeout=180,
            headers={"User-Agent": "OpenMontage/1.0"},
        ) as r:
            r.raise_for_status()
            with open(out_path, "wb") as f:
                for chunk in r.iter_content(chunk_size=1 << 16):
                    if chunk:
                        f.write(chunk)
        return out_path

View on GitHub (pinned to 95e1c3d0ab)

Solutions

1. Check __cause__ to separate scraping failures from HTTP/timeout failures.
2. Make ESA per-clip failures non-fatal in batch pipelines: catch, log, backfill from Archive.org or Pexels.
3. For timeouts, download large ESA assets directly with a resumable downloader (curl -C -, wget -c) using the same URL.
4. Refresh search results rather than reusing old candidates whose URLs may have expired.

Example fix

# before
path = esa.download(candidate, out)

# after
try:
    path = esa.download(candidate, out)
except RuntimeError as e:
    logger.warning("esa clip %s failed: %s", candidate.clip_id, e.__cause__ or e)
    path = backfill_from_alternate_source(candidate, out)

for c in candidates:
    try:
        paths.append(esa.download(c, out))
    except RuntimeError as e:
        logger.warning("esa %s failed (%s), continuing", c.clip_id, e.__cause__ or e)
        continue

When it happens

Trigger: Expired or moved esa.int asset URLs returning 404, 403 from the ESA CDN, requests.Timeout on slow multi-hundred-MB space footage, or any parse exception while building the soup — all surface as this RuntimeError.

Common situations: Long-running corpus builders where ESA links rot over months; large-format mission footage exceeding the 180s stream timeout on slow links; intermittent CDN blocks of non-browser user agents.

Related errors

• Dareful download failed for {detail_url}: {e}
• Could not find download URL on ESA detail page: {detail_url}
• Candidate {candidate.clip_id} has no download_url
• Could not find download URL on Dareful page: {detail_url}
• Downloading Atlas Cloud output failed: {exc}

AI-assisted analysis of calesthio/OpenMontage@95e1c3d0ab (2026-08-15). Data as JSON: /api/errors/fded6d607fc1a360. Report an issue: GitHub.