unclecode/crawl4ai · warning · TimeoutError

Timeout after {timeout}ms waiting for selector '{css_selecto

Error message

Timeout after {timeout}ms waiting for selector '{css_selector}'

What it means

ArtifactTooLarge is raised by write_artifact when a single artifact's byte length exceeds MAX_ARTIFACT_BYTES (env CRAWL4AI_MAX_ARTIFACT_BYTES, default 50 MiB). It is a hard per-item cap enforced before the store is even initialized, so oversized data never touches disk.

Source

Thrown at crawl4ai/async_crawler_strategy.py:265

            timeout (float): Maximum time to wait in milliseconds

        Returns:
            None
        """
        wait_for = wait_for.strip()

        if wait_for.startswith("js:"):
            # Explicitly specified JavaScript
            js_code = wait_for[3:].strip()
            return await self.csp_compliant_wait(page, js_code, timeout)
        elif wait_for.startswith("css:"):
            # Explicitly specified CSS selector
            css_selector = wait_for[4:].strip()
            try:
                await page.wait_for_selector(css_selector, timeout=timeout)
            except Error as e:
                if "Timeout" in str(e):
                    raise TimeoutError(
                        f"Timeout after {timeout}ms waiting for selector '{css_selector}'"
                    )
                else:
                    raise ValueError(f"Invalid CSS selector: '{css_selector}'")
        else:
            # Auto-detect based on content
            if wait_for.startswith("()") or wait_for.startswith("function"):
                # It's likely a JavaScript function
                return await self.csp_compliant_wait(page, wait_for, timeout)
            else:
                # Assume it's a CSS selector first
                try:
                    await page.wait_for_selector(wait_for, timeout=timeout)
                except Error as e:
                    if "Timeout" in str(e):
                        raise TimeoutError(
                            f"Timeout after {timeout}ms waiting for selector '{wait_for}'"
                        )

View on GitHub (pinned to 7e80152142)

Solutions

1. Reduce artifact size at the source (lower screenshot resolution/viewport, cap PDF page count, trim content)
2. Raise CRAWL4AI_ARTIFACT quota env is NOT the fix — raise CRAWL4AI_MAX_ARTIFACT_BYTES on the server if larger single artifacts are legitimate
3. Split the artifact into chunks below the per-item limit if the server limit must stay

Example fix

# server: raise per-artifact cap if large PDFs are expected
# docker-compose.yml
environment:
  CRAWL4AI_MAX_ARTIFACT_BYTES: "104857600"  # 100 MiB

MAX_ARTIFACT_BYTES = int(os.environ.get("CRAWL4AI_MAX_ARTIFACT_BYTES", 50 * 1024 * 1024))
def safe_artifact(data: bytes) -> bool:
    return len(data) <= MAX_ARTIFACT_BYTES

from artifacts import write_artifact, ArtifactTooLarge
try:
    meta = write_artifact("screenshot", data)
except ArtifactTooLarge:
    data = shrink(data)  # lower resolution / trim
    meta = write_artifact("screenshot", data)

When it happens

Trigger: Calling the artifact API (e.g. POST /artifact or an endpoint that stores screenshots/PDFs via write_artifact) with a body whose encoded size > 50 MiB (or your configured limit). len(data) is checked directly against the constant.

Common situations: Full-page screenshots or PDFs of very long pages; raising crawl quality/resolution so media blobs grow past the default; lowering CRAWL4AI_MAX_ARTIFACT_BYTES in a memory-tight deployment and forgetting old payloads exceed it.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Invalid CSS selector: '{css_selector}'
• Artifact too large
• Timeout after {timeout}ms waiting for selector '{wait_for}'
• Invalid wait_for parameter: '{wait_for}'. It should be eithe
• Invalid config

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/3c2812b4cfeba179. Report an issue: GitHub.