unclecode/crawl4ai · error · RuntimeError

Failed on navigating ACS-GOTO: {str(e)}

Error message

Failed on navigating ACS-GOTO:
{str(e)}

What it means

Raised when Playwright's page.goto() raises an Error during navigation that is not the expected 'net::ERR_ABORTED' download-abort case. The underlying Playwright exception (DNS failure, TLS error, timeout, net::ERR_CONNECTION_REFUSED, etc.) is stringified and wrapped in a RuntimeError with the ACS-GOTO tag, aborting the crawl of that URL.

Source

Thrown at crawl4ai/async_crawler_strategy.py:778

                            )

                        response = await page.goto(
                            url, wait_until=config.wait_until, timeout=config.page_timeout
                        )
                        redirected_url = page.url
                        redirected_status_code = response.status if response else None
                    except Error as e:
                        # Allow navigation to be aborted when downloading files
                        # This is expected behavior for downloads in some browser engines
                        if 'net::ERR_ABORTED' in str(e) and self.browser_config.accept_downloads:
                            self.logger.info(
                                message=f"Navigation aborted, likely due to file download: {url}",
                                tag="GOTO",
                                params={"url": url},
                            )
                            response = None
                        else:
                            raise RuntimeError(f"Failed on navigating ACS-GOTO:\n{str(e)}")

                    # ──────────────────────────────────────────────────────────────
                    # Walk the redirect chain.  Playwright returns only the last
                    # hop, so we trace the `request.redirected_from` links until the
                    # first response that differs from the final one and surface its
                    # status-code.
                    # ──────────────────────────────────────────────────────────────
                    if response is None:
                        status_code = 200
                        response_headers = {}
                    else:
                        first_resp = response
                        req = response.request
                        while req and req.redirected_from:
                            prev_req = req.redirected_from
                            prev_resp = await prev_req.response()
                            if prev_resp:                       # keep earliest
                                first_resp = prev_resp

View on GitHub (pinned to 7e80152142)

Solutions

  1. Read the embedded Playwright error text to identify the root cause (DNS vs TLS vs timeout) and fix that specifically.
  2. For slow pages, increase CrawlerRunConfig.page_timeout; for TLS issues set BrowserConfig(ignore_https_errors=True) or install the CA cert.
  3. For download-triggering pages that abort navigation, set BrowserConfig(accept_downloads=True) so ERR_ABORTED is treated as expected.
  4. For transient network issues, retry the URL with backoff at the caller level.

Example fix

// before
cfg = CrawlerRunConfig(page_timeout=10000)
result = await crawler.arun(url=url, config=cfg)

// after
cfg = CrawlerRunConfig(page_timeout=60000)
browser_cfg = BrowserConfig(ignore_https_errors=True)
async with AsyncWebCrawler(config=browser_cfg) as crawler:
    try:
        result = await crawler.arun(url=url, config=cfg)
    except RuntimeError as e:
        logger.warning(f"goto failed for {url}: {e}")
Defensive patterns

Strategy: retry

Validate before calling

import socket
from urllib.parse import urlparse

def host_reachable(url: str) -> bool:
    h = urlparse(url).hostname
    try:
        socket.getaddrinfo(h or "", 80)
        return True
    except socket.gaierror:
        return False

Try / catch

try:
    result = await crawler.arun(url, config=cfg)
except RuntimeError as e:
    msg = str(e)
    if "ERR_NAME_NOT_RESOLVED" in msg or "ERR_CONNECTION_REFUSED" in msg:
        mark_dead(url)
    elif "Timeout" in msg:
        await asyncio.sleep(2)
        result = await crawler.arun(url, config=cfg)

Prevention

When it happens

Trigger: Any page.goto() failure: unreachable host (ERR_NAME_NOT_RESOLVED), refused connection, certificate errors with ignore_https_errors off, navigation exceeding config.page_timeout (TimeoutError from Playwright), or a target that resets the connection. Downloads abort with ERR_ABORTED are exempt only when browser_config.accept_downloads is true.

Common situations: Corporate proxies blocking direct egress; sites with invalid/self-signed TLS certs; slow pages exceeding page_timeout; transient DNS failures in CI; crawling localhost ports where the dev server is not running.

Related errors

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/4e00e01037c2ec37. Report an issue: GitHub.