unclecode/crawl4ai · error · RuntimeError

Wait condition failed: {str(e)}

Error message

Wait condition failed: {str(e)}

What it means

Raised when the user-supplied wait_for condition (a CSS selector, JS function string, or js_condition) fails or times out inside smart_wait(). The timeout used is config.wait_for_timeout if set, otherwise config.page_timeout; the underlying exception text is wrapped in a RuntimeError with the 'Wait condition failed' prefix.

Source

Thrown at crawl4ai/async_crawler_strategy.py:994

            # Handle user simulation — generate mouse movement and scroll
            # signals that anti-bot systems look for, without firing keyboard
            # events (ArrowDown triggers JS framework navigation) or clicking
            # at fixed positions (may hit buttons/links and navigate away).
            if config.simulate_user or config.magic:
                await page.mouse.move(random.randint(100, 300), random.randint(150, 300))
                await page.mouse.move(random.randint(300, 600), random.randint(200, 400))
                await page.mouse.wheel(0, random.randint(200, 400))

            # --- Phase 2: Wait for page readiness ---

            if config.wait_for:
                try:
                    timeout = config.wait_for_timeout if config.wait_for_timeout is not None else config.page_timeout
                    await self.smart_wait(
                        page, config.wait_for, timeout=timeout
                    )
                except Exception as e:
                    raise RuntimeError(f"Wait condition failed: {str(e)}")

            # Handle virtual scroll if configured (after wait_for so container exists)
            if config.virtual_scroll_config:
                await self._handle_virtual_scroll(page, config.virtual_scroll_config)

            # Pre-content retrieval hooks and delay
            await self.execute_hook("before_retrieve_html", page, context=context, config=config)
            if config.delay_before_return_html:
                await asyncio.sleep(config.delay_before_return_html)

            # --- Phase 3: Post-wait JS (runs on fully-loaded page) ---

            if config.js_code:
                execution_result = await self.robust_execute_user_script(
                    page, config.js_code
                )

                if not execution_result["success"]:

View on GitHub (pinned to 7e80152142)

Solutions

  1. Verify the selector/JS condition manually in browser devtools against the exact crawled URL.
  2. Increase wait_for_timeout (or page_timeout) to cover slow rendering.
  3. Make the condition resilient: wait for a selector that always exists once content is meaningful, or use 'js:' conditions that poll document readiness.
  4. Catch the RuntimeError per-URL when crawling many pages so one bad page does not abort the batch.

Example fix

// before
cfg = CrawlerRunConfig(wait_for="css:.search-results", wait_for_timeout=3000)

// after
cfg = CrawlerRunConfig(
    wait_for="css:.search-results, .no-results",
    wait_for_timeout=15000,
)
Defensive patterns

Strategy: try-catch

Validate before calling

async def selector_present(browser_page, css: str, timeout_ms=2000) -> bool:
    try:
        await browser_page.wait_for_selector(css, timeout=timeout_ms, state="attached")
        return True
    except Exception:
        return False

Try / catch

try:
    result = await crawler.arun(url, config=cfg)
except RuntimeError as e:
    if "Wait condition failed" in str(e):
        result = await crawler.arun(url, config=CrawlerRunConfig(wait_for=None))  # degrade gracefully

Prevention

When it happens

Trigger: Setting CrawlerRunConfig(wait_for='css:.results', wait_for_timeout=5000) on a page where .results never appears within the timeout; a wait_for js: expression that throws or returns falsy forever; a selector that appears only after interaction (click, scroll, login).

Common situations: Selector names change after a site redesign; A/B-tested pages where the selector sometimes never renders; timeout too short for slow SPAs or rate-limited APIs backing the page; wait_for expression syntax errors (invalid JS).

Related errors

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/61ee8d021d4b43d7. Report an issue: GitHub.