unclecode/crawl4ai · error · ConnectionError

Failed to connect: {str(e)}

Error message

Failed to connect: {str(e)}

What it means

Raised by Crawl4aiDockerClient._request when the request fails with httpx.RequestError (excluding timeouts, which are handled first) — connection refused, DNS failure, TLS errors, or a reset connection. The httpx detail string is embedded in ConnectionError('Failed to connect: ...').

Source

Thrown at crawl4ai/docker_client.py:119

            request_data["hooks"] = {
                "code": hooks_code,
                "timeout": hooks_timeout
            }

        return request_data

    async def _request(self, method: str, endpoint: str, **kwargs) -> httpx.Response:
        """Make an HTTP request with error handling."""
        url = urljoin(self.base_url, endpoint)
        try:
            response = await self._http_client.request(method, url, **kwargs)
            response.raise_for_status()
            return response
        except httpx.TimeoutException as e:
            raise ConnectionError(f"Request timed out: {str(e)}")
        except httpx.RequestError as e:
            raise ConnectionError(f"Failed to connect: {str(e)}")
        except httpx.HTTPStatusError as e:
            error_msg = (e.response.json().get("detail", str(e)) 
                        if "application/json" in e.response.headers.get("content-type", "") 
                        else str(e))
            raise RequestError(f"Server error {e.response.status_code}: {error_msg}")

    async def crawl(
        self,
        urls: List[str],
        browser_config: Optional[BrowserConfig] = None,
        crawler_config: Optional[CrawlerRunConfig] = None,
        hooks: Optional[Union[Dict[str, Callable], Dict[str, str]]] = None,
        hooks_timeout: int = 30
    ) -> Union[CrawlResult, List[CrawlResult], AsyncGenerator[CrawlResult, None]]:
        """
        Execute a crawl operation.

        Args:

View on GitHub (pinned to 7e80152142)

Solutions

  1. Re-check server health: GET {base_url}/health; restart the container if it is down
  2. Read the embedded httpx detail: 'Connection refused' = server not listening on that port; TLS errors = certificate/scheme mismatch
  3. Wrap calls in retry-with-reconnect logic for long-running sessions, recreating the client after server restarts
  4. Stabilize the server (memory limits, restart policy) if it dies under load

Example fix

// before
results = await client.crawl(urls, ...)  # mid-session server death

// after
try:
    results = await client.crawl(urls, ...)
except ConnectionError as e:
    if "Failed to connect" in str(e):
        await client.close()
        client = Crawl4aiDockerClient(base_url=BASE_URL)
        await client.authenticate(EMAIL)
        results = await client.crawl(urls, ...)
Defensive patterns

Strategy: retry

Validate before calling

import httpx

async def reachable(base_url: str) -> bool:
    try:
        await httpx.AsyncClient().get(f"{base_url}/health", timeout=5)
        return True
    except httpx.HTTPError:
        return False

Try / catch

async def crawl_resilient(make_client, **kw):
    for i in range(3):
        client = make_client()
        try:
            return await client.crawl(**kw)
        except ConnectionError as e:
            if "Failed to connect" not in str(e) or i == 2:
                raise
            await asyncio.sleep(2 ** i)  # server may be restarting

Prevention

When it happens

Trigger: Any client call made after the server container has stopped or restarted mid-session; base_url host unreachable mid-run; TLS certificate mismatch when using https; connection reset by a proxy.

Common situations: The Docker server exits (OOM, crash) while a batch of crawls is in flight; a laptop sleeping/restoring network mid-run; Kubernetes pod restart; the httpx client holding a pooled connection to a now-dead server.

Related errors

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/b3d9e60d458da1c0. Report an issue: GitHub.