{"record":{"id":"aa2a25b6943b976a","repo":"unclecode/crawl4ai","slug":"e","errorCode":null,"errorMessage":"{e}","messagePattern":"\\{e\\}","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"crawl4ai/async_webcrawler.py:789","lineNumber":1850,"sourceCode":"            params = config.__dict__.copy()\n            params.pop(\"url\", None)\n            # add keys from kwargs to params that doesn't exist in params\n            params.update({k: v for k, v in kwargs.items()\n                          if k not in params.keys()})\n\n            ################################\n            # Scraping Strategy Execution  #\n            ################################\n            result: ScrapingResult = scraping_strategy.scrap(\n                url, html, **params)\n\n            if result is None:\n                raise ValueError(\n                    f\"Process HTML, Failed to extract content from the website: {url}\"\n                )\n\n        except InvalidCSSSelectorError as e:\n            raise ValueError(str(e))\n        except Exception as e:\n            raise ValueError(\n                f\"Process HTML, Failed to extract content from the website: {url}, error: {str(e)}\"\n            )\n\n        # Extract results - handle both dict and ScrapingResult\n        if isinstance(result, dict):\n            cleaned_html = sanitize_input_encode(\n                result.get(\"cleaned_html\", \"\"))\n            media = result.get(\"media\", {})\n            links = result.get(\"links\", {})\n            metadata = result.get(\"metadata\", {})\n        else:\n            cleaned_html = sanitize_input_encode(result.cleaned_html)\n            media = result.media.model_dump()\n            links = result.links.model_dump()\n            metadata = result.metadata\n","sourceCodeStart":1832,"sourceCodeEnd":1868,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/deploy/docker/c4ai-code-context.md#L1832-L1868","documentation":"ValueError re-raise from AsyncWebCrawler.process_html: an InvalidCSSSelectorError escaped the scraping strategy and is re-raised verbatim (str(e)) - no URL is attached, so the message is exactly the selector error text.","triggerScenarios":"config.css_selector, content_selector, or exclude_css contains an invalid CSS selector such as 'a[href', '>>div', or a stray pseudo-class the parser rejects; the strategy validates selectors and throws InvalidCSSSelectorError during scrap().","commonSituations":"Hand-written selectors with typos; selectors generated from templates containing empty fragments; using Playwright/JS-style selectors (':has-text(...)') where a CSS engine is used; changing exclude_css values between runs without re-validating.","solutions":["Test each selector locally: BeautifulSoup('<div></div>', 'html.parser').select(your_selector) must not raise","Quote attribute values and balance brackets: a[href^=\"https://\"] not a[href^=https://]","If you need Playwright-specific selectors, pass them via js_code/wait_for rather than css_selector","Wrap arun in try/except ValueError and log which URL/selector combo failed when batch-crawling"],"exampleFix":"# before\nconfig = CrawlerRunConfig(css_selector='article >> .content')\n\n# after\nconfig = CrawlerRunConfig(css_selector='article .content')","handlingStrategy":"validation","validationCode":"from bs4 import BeautifulSoup\n\ndef selectors_valid(*selectors) -> bool:\n    probe = BeautifulSoup(\"<div><a href='#'>x</a></div>\", \"html.parser\")\n    try:\n        for s in selectors:\n            if s:\n                probe.select(s)\n        return True\n    except Exception:\n        return False","typeGuard":"def is_valid_css_selector(s) -> bool:\n    if not s:\n        return True\n    try:\n        BeautifulSoup(\"<div></div>\", \"html.parser\").select(s)\n        return True\n    except Exception:\n        return False","tryCatchPattern":"try:\n    await crawler.arun(url=url, config=config)\nexcept ValueError as e:\n    msg = str(e)\n    if \"selector\" in msg.lower() or \"invalid\" in msg.lower():\n        config = CrawlerRunConfig(**{**vars_like(config), \"css_selector\": fix_selector(config.css_selector)})\n    else:\n        raise","preventionTips":["Validate css_selector/exclude_css values in a unit test whenever they change","Keep selectors plain CSS; route engine-specific selectors through js_code/wait_for"],"tags":["crawl4ai","css-selector","validation","scraping"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}