sigoden/aichat · error · anyhow::Error

Invalid extract selector

Error message

Invalid extract selector, {}

What it means

When crawl options include an `extract` CSS selector, `crawl_page` parses it with `scraper::Selector::parse` and returns `anyhow!("Invalid extract selector, {}", err)` if parsing fails. The error includes the underlying selector-syntax error, so the selector used for extracting text from the crawled HTML is invalid.

Solutions

1. Validate the selector in the browser devtools (`document.querySelector(...)`) before putting it in config.
2. Fix syntax errors reported in the message: unbalanced brackets, quotes, or parentheses.
3. Convert XPath/jQuery expressions to CSS selector syntax.
4. Pre-test with `Selector::parse(...)` at config-load time to fail fast with a clear message.

Example fix

// before
let options = CrawlOptions { extract: Some("//article[@class='post']".into()), ..Default::default() };
// after
let options = CrawlOptions { extract: Some("article.post".into()), ..Default::default() };

fn validate_selector(sel: &str) -> Result<(), String> {
    match scraper::Selector::parse(sel) {
        Ok(_) => Ok(()),
        Err(e) => Err(format!("invalid extract selector {sel:?}: {e}")),
    }
}
// call at config load: validate_selector(options.extract.as_deref().unwrap_or("*"))?;

match crawl_website(&url, options).await {
    Err(e) if e.to_string().starts_with("Invalid extract selector") => {
        eprintln!("Fix --extract: {e}");
    }
    other => other?,
}

When it happens

Trigger: Passing a syntactically invalid CSS selector in `options.extract` to `crawl_website`/`crawl_page`, e.g. `div[foo=` (unclosed attribute), empty string, or unsupported pseudo-classes.

Common situations: Hand-written selectors in config with typos or unbalanced brackets/quotes; selectors copied from XPath or jQuery that aren't valid CSS (e.g. `//div[@id]`); scraper crate not supporting certain pseudo-elements.

Related errors

• Invalid crawl page at
• Invalid messages
• Failed to parse '.tools
• Unexpected media type
• Invalid gh tree

AI-assisted analysis of sigoden/aichat@82976d349a (2026-09-09). Data as JSON: /api/errors/73e77b34983d7719. Report an issue: GitHub.

Appendix: source

Thrown at src/utils/request.rs:414

    for element in document.select(&selector) {
        if let Some(href) = element.value().attr("href") {
            let href = Url::parse(href).ok().or_else(|| location.join(href).ok());
            match href {
                None => continue,
                Some(href) => {
                    if href.as_str().starts_with(location.as_str())
                        && !should_exclude_link(href.path(), &options.exclude)
                    {
                        links.insert(href.path().to_string());
                    }
                }
            }
        }
    }

    let text = if let Some(selector) = &options.extract {
        let selector = Selector::parse(selector)
            .map_err(|err| anyhow!("Invalid extract selector, {}", err))?;
        document
            .select(&selector)
            .map(|v| html_to_md(&v.html()))
            .collect::<Vec<String>>()
            .join("\n\n")
    } else {
        html_to_md(&body)
    };

    Ok((path.to_string(), text, links.into_iter().collect()))
}

fn should_exclude_link(link: &str, exclude: &[String]) -> bool {
    if link.contains("#") {
        return true;
    }
    let parts: Vec<&str> = link.trim_end_matches('/').split('/').collect();
    let name = parts.last().unwrap_or(&"").to_lowercase();

View on GitHub (pinned to 82976d349a)