unclecode/crawl4ai · error · Error

Body element is hidden: {visibility_info}

Error message

Body element is hidden: {visibility_info}

What it means

Raised when the crawled page's <body> element is detected as hidden (display:none, visibility:hidden, opacity:0, or zero size) after the Playwright wait_for_selector on body times out or errors. The visibility details from check_visibility(page) are embedded in the message. It is skipped entirely when config.ignore_body_visibility is true.

Source

Thrown at crawl4ai/async_crawler_strategy.py:844

                    timeout=30000,
                )

                if not is_visible and not config.ignore_body_visibility:
                    visibility_info = await self.check_visibility(page)
                    raise Error(f"Body element is hidden: {visibility_info}")

            except Error:
                visibility_info = await self.check_visibility(page)

                if self.browser_config.verbose:
                    self.logger.debug(
                        message="Body visibility info: {info}",
                        tag="DEBUG",
                        params={"info": visibility_info},
                    )

                if not config.ignore_body_visibility:
                    raise Error(f"Body element is hidden: {visibility_info}")

            # try:
            #     await page.wait_for_selector("body", state="attached", timeout=30000)

            #     await page.wait_for_function(
            #         """
            #         () => {
            #             const body = document.body;
            #             const style = window.getComputedStyle(body);
            #             return style.display !== 'none' &&
            #                 style.visibility !== 'hidden' &&
            #                 style.opacity !== '0';
            #         }
            #     """,
            #         timeout=30000,
            #     )
            # except Error as e:
            #     visibility_info = await page.evaluate(

View on GitHub (pinned to 7e80152142)

Solutions

  1. Set CrawlerRunConfig(ignore_body_visibility=True) when the target is known to hide body initially.
  2. Add wait_for / js_code to wait until the app finishes rendering (e.g. wait for a specific selector) before visibility is checked.
  3. Increase page_timeout so hydration completes; use wait_until='networkidle' or 'domcontentloaded' as appropriate.
  4. If the site blocks headless browsers, adjust the user agent or use an appropriate adapter.

Example fix

// before
cfg = CrawlerRunConfig()
result = await crawler.arun(url=spa_url, config=cfg)

// after
cfg = CrawlerRunConfig(
    ignore_body_visibility=True,
    wait_for="css:.app-root[data-ready='true']",
)
result = await crawler.arun(url=spa_url, config=cfg)
Defensive patterns

Strategy: fallback

Try / catch

try:
    result = await crawler.arun(url, config=cfg)
except Exception as e:
    if "Body element is hidden" in str(e):
        cfg2 = CrawlerRunConfig(**{**cfg.__dict__, "ignore_body_visibility": True})
        result = await crawler.arun(url, config=cfg2)

Prevention

When it happens

Trigger: Pages that keep body hidden until JS runs (loading screens, cookie walls, SPA shells, geo-block pages, pages that render only after user interaction). Also pages where an overlay CSS hides body, or where content is inside frames/containers while body itself never becomes visible.

Common situations: SPAs (React/Vue/Angular) with slow hydration hiding body behind a loader; sites that block bots by leaving body hidden; pages requiring cookies/login before revealing content; headless-mode-specific CSS (some sites hide body for headless user agents).

Related errors

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/1cdb63b7eb2976f7. Report an issue: GitHub.