unclecode/crawl4ai · error · ConnectionError

Request timed out: {str(e)}

Error message

Request timed out: {str(e)}

What it means

Raised by Crawl4aiDockerClient._request when the httpx call to the server raises httpx.TimeoutException — the request exceeded the configured timeout before completing. It is converted to ConnectionError('Request timed out: ...') with the underlying httpx timeout detail.

Source

Thrown at crawl4ai/docker_client.py:117

                # Already in string format
                hooks_code = hooks

            request_data["hooks"] = {
                "code": hooks_code,
                "timeout": hooks_timeout
            }

        return request_data

    async def _request(self, method: str, endpoint: str, **kwargs) -> httpx.Response:
        """Make an HTTP request with error handling."""
        url = urljoin(self.base_url, endpoint)
        try:
            response = await self._http_client.request(method, url, **kwargs)
            response.raise_for_status()
            return response
        except httpx.TimeoutException as e:
            raise ConnectionError(f"Request timed out: {str(e)}")
        except httpx.RequestError as e:
            raise ConnectionError(f"Failed to connect: {str(e)}")
        except httpx.HTTPStatusError as e:
            error_msg = (e.response.json().get("detail", str(e)) 
                        if "application/json" in e.response.headers.get("content-type", "") 
                        else str(e))
            raise RequestError(f"Server error {e.response.status_code}: {error_msg}")

    async def crawl(
        self,
        urls: List[str],
        browser_config: Optional[BrowserConfig] = None,
        crawler_config: Optional[CrawlerRunConfig] = None,
        hooks: Optional[Union[Dict[str, Callable], Dict[str, str]]] = None,
        hooks_timeout: int = 30
    ) -> Union[CrawlResult, List[CrawlResult], AsyncGenerator[CrawlResult, None]]:
        """
        Execute a crawl operation.

View on GitHub (pinned to 7e80152142)

Solutions

1. Raise the timeout for slow operations: pass a larger hooks_timeout to crawl(...)
2. Construct the client with a larger default httpx timeout if the base client exposes it
3. Reduce crawl size (fewer URLs per call) or split into batches so each request finishes faster
4. Check server load/logs — a systematically slow server needs resources, not just bigger timeouts

Example fix

// before
results = await client.crawl(urls, browser_config=b, crawler_config=c)  # 30s default

// after
results = await client.crawl(urls, browser_config=b, crawler_config=c, hooks_timeout=300)

import asyncio

async def crawl_with_retry(client, **kw):
    for i in range(3):
        try:
            return await client.crawl(**kw)
        except ConnectionError as e:
            if "timed out" not in str(e) or i == 2:
                raise
            await asyncio.sleep(2 ** i)
            kw["hooks_timeout"] = kw.get("hooks_timeout", 30) * 2

When it happens

Trigger: Any client API call (crawl, get_schema, etc.) where the server takes longer than the effective httpx timeout — note crawl() passes hooks_timeout (default 30s) as the request timeout; long crawls with on-hook execution can exceed it.

Common situations: Crawling slow or large sites through the Docker server with the default 30-second timeout; server under heavy load; hooks_timeout left at default while browser hooks (login, waits) take minutes; running many concurrent crawl batches that queue on the server.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Failed on navigating ACS-GOTO: {str(e)}
• Request timed out: {str(e)}
• Authentication failed: {str(e)}
• Cannot connect to server: {str(e)}
• Failed to connect: {str(e)}

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/32850911db420b00. Report an issue: GitHub.