unclecode/crawl4ai · warning · ValueError

Invalid CSS selector: '{css_selector}'

Error message

Invalid CSS selector: '{css_selector}'

What it means

QuotaExceeded is raised by write_artifact when adding this artifact would push the total on-disk size of the artifact store past ARTIFACT_QUOTA_BYTES (env CRAWL4AI_ARTIFACT_QUOTA_BYTES, default 2 GiB). Unlike the per-item cap, this is a shared budget across all stored artifacts; expired artifacts are reaped by a janitor, so the quota is recoverable.

Source

Thrown at crawl4ai/async_crawler_strategy.py:269

        """
        wait_for = wait_for.strip()

        if wait_for.startswith("js:"):
            # Explicitly specified JavaScript
            js_code = wait_for[3:].strip()
            return await self.csp_compliant_wait(page, js_code, timeout)
        elif wait_for.startswith("css:"):
            # Explicitly specified CSS selector
            css_selector = wait_for[4:].strip()
            try:
                await page.wait_for_selector(css_selector, timeout=timeout)
            except Error as e:
                if "Timeout" in str(e):
                    raise TimeoutError(
                        f"Timeout after {timeout}ms waiting for selector '{css_selector}'"
                    )
                else:
                    raise ValueError(f"Invalid CSS selector: '{css_selector}'")
        else:
            # Auto-detect based on content
            if wait_for.startswith("()") or wait_for.startswith("function"):
                # It's likely a JavaScript function
                return await self.csp_compliant_wait(page, wait_for, timeout)
            else:
                # Assume it's a CSS selector first
                try:
                    await page.wait_for_selector(wait_for, timeout=timeout)
                except Error as e:
                    if "Timeout" in str(e):
                        raise TimeoutError(
                            f"Timeout after {timeout}ms waiting for selector '{wait_for}'"
                        )
                    else:
                        # If it's not a timeout error, it might be an invalid selector
                        # Let's try to evaluate it as a JavaScript function as a fallback
                        try:

View on GitHub (pinned to 7e80152142)

Solutions

  1. Trigger or wait for the janitor/TTL sweep (expired artifacts are unlinked, freeing quota) and retry
  2. Raise CRAWL4AI_ARTIFACT_QUOTA_BYTES if the volume is legitimate and disk allows
  3. Lower CRAWL4AI_TTL (CRAWL4AI_ARTIFACT_TTL_SECONDS) so artifacts age out faster under sustained load

Example fix

# server: free quota via janitor endpoint/call, or raise budget
# docker-compose.yml
environment:
  CRAWL4AI_ARTIFACT_QUOTA_BYTES: "5368709120"  # 5 GiB
  CRAWL4AI_ARTIFACT_TTL_SECONDS: "1800"
Defensive patterns

Strategy: retry

Try / catch

from artifacts import write_artifact, QuotaExceeded
try:
    meta = write_artifact(kind, data)
except QuotaExceeded:
    janitor()  # reap expired artifacts to free budget
    meta = write_artifact(kind, data)  # one bounded retry

Prevention

When it happens

Trigger: Storing an artifact when _dir_size() + len(data) > 2 GiB (or configured quota) — e.g. sustained crawl runs generating screenshots/PDFs until the store fills, with the TTL (default 1 h) not yet expiring old entries.

Common situations: High-throughput batch crawls producing many media artifacts; a low quota set for a small disk; janitor not running often enough so expired artifacts still count toward the measured size.

Related errors

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/830bc2ac9de95769. Report an issue: GitHub.