unclecode/crawl4ai · error · InvalidCSSSelectorError

Invalid CSS selector: {css_selector}

Error message

Invalid CSS selector: {css_selector}

What it means

This InvalidCSSSelectorError (crawl4ai/utils.py:1140) is a catch-all re-raise at the end of an except Exception block in the HTML-processing helper: ANY exception raised while cleaning/extracting content (not just selector failures) is printed and re-raised as InvalidCSSSelectorError with the chained cause. So the real failure may be unrelated to CSS selectors — inspect the 'from e' __cause__ and the printed 'Error processing HTML content:' line.

Source

Thrown at crawl4ai/utils.py:1140

        try:
            meta = extract_metadata(html, soup)
        except Exception as e:
            print("Error extracting metadata:", str(e))
            meta = {}

        # Return the Markdown content
        return {
            "markdown": markdown,
            "cleaned_html": cleaned_html,
            "success": True,
            "media": media,
            "links": links,
            "metadata": meta,
        }

    except Exception as e:
        print("Error processing HTML content:", str(e))
        raise InvalidCSSSelectorError(f"Invalid CSS selector: {css_selector}") from e


def get_content_of_website_optimized(
    url: str,
    html: str,
    word_count_threshold: int = MIN_WORD_THRESHOLD,
    css_selector: str = None,
    **kwargs,
) -> Dict[str, Any]:
    """
    Extracts and cleans content from website HTML, optimizing for useful media and contextual information.
    
    Parses the provided HTML to extract internal and external links, filters and scores images for usefulness, gathers contextual descriptions for media, removes unwanted or low-value elements, and converts the cleaned HTML to Markdown. Also extracts metadata and returns all structured content in a dictionary.
    
    Args:
        url: The URL of the website being processed.
        html: The raw HTML content to extract from.
        word_count_threshold: Minimum word count for elements to be retained.

View on GitHub (pinned to 7e80152142)

Solutions

1. Read the full traceback and the chained cause (raise ... from e) — the printed 'Error processing HTML content:' message names the real exception.
2. If the cause is a SelectorSyntaxError, replace Playwright/Selenium-specific selector syntax with standard CSS supported by BeautifulSoup (soupsieve).
3. If the cause is the no-elements case, fix the selector per error 140 guidance.
4. If the cause is malformed HTML, pre-clean with BeautifulSoup(html, 'html.parser').prettify() or pass a parser that tolerates the input.

Example fix

// before
try:
    out = process_html(html, css_selector="a >> text=foo")
except InvalidCSSSelectorError:
    pass  # real cause hidden

// after
try:
    out = process_html(html, css_selector="a.foo")
except InvalidCSSSelectorError as e:
    logging.error("underlying cause: %r", e.__cause__)

try:
    out = process_html(html, css_selector=sel)
except InvalidCSSSelectorError as e:
    cause = e.__cause__
    if "SelectorSyntaxError" in type(cause).__name__:
        fix_selector()      # unsupported pseudo-class syntax
    else:
        inspect(cause)      # real error is chained, not selector-related

When it happens

Trigger: Any exception inside the try block of this helper: a css_selector that triggers a BeautifulSoup SelectorSyntaxError; unexpected None from a malformed document causing an AttributeError; or the underlying no-elements match condition — all surface as this same error. Calling the function with an exotic selector like ":has-text('foo')" (not supported by BeautifulSoup) hits it immediately.

Common situations: Developers assume the error means the selector is wrong and rewrite it repeatedly when the actual cause (visible in the chained exception) is something else entirely, e.g. a NoneType error or an unsupported pseudo-class; using Playwright-style selectors (::text, >>, :has-text) with the BeautifulSoup-based utility.

Related errors

• Invalid CSS selector , No elements found for CSS selector: {
• Invalid CSS selector, No elements found for CSS selector: {c
• {e}
• Failed to extract HTML content: {str(e)}
• This function must be run in Google Colab environment.

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/7a77811e059fed77. Report an issue: GitHub.