dgtlmoon/changedetection.io · error · Exception

Content Fetcher > xPath scraper failed. Please report this U

Error message

Content Fetcher > xPath scraper failed. Please report this URL so we can fix it :)

What it means

Generic Exception raised when page.evaluate of the xpath-element scraper JS returns falsy — the in-page xpath extraction script produced no data, treated as an internal scraper failure the user is asked to report.

Source

Thrown at changedetectionio/content_fetchers/puppeteer.py:498

            await self.page.evaluate(f"var include_filters=''")

        MAX_TOTAL_HEIGHT = int(os.getenv("SCREENSHOT_MAX_HEIGHT", SCREENSHOT_MAX_HEIGHT_DEFAULT))

        self.content = await self.page.content

        # Now take screenshot (scrolling may trigger layout changes, but measurements are already captured)
        logger.debug(f"Screenshot format {self.screenshot_format}")
        self.screenshot = await capture_full_page(page=self.page, screenshot_format=self.screenshot_format, watch_uuid=watch_uuid, lock_viewport_elements=self.lock_viewport_elements)

        # Force garbage collection - pyppeteer base64 decode creates temporary buffers
        import gc
        gc.collect()
        self.xpath_data = await self.page.evaluate(XPATH_ELEMENT_JS, {
            "visualselector_xpath_selectors": visualselector_xpath_selectors,
            "max_height": MAX_TOTAL_HEIGHT
        })
        if not self.xpath_data:
            raise Exception(f"Content Fetcher > xPath scraper failed. Please report this URL so we can fix it :)")


        self.instock_data = await self.page.evaluate(INSTOCK_DATA_JS)

        # It's good to log here in the case that the browser crashes on shutting down but we still get the data we need
        logger.success(f"Fetching '{url}' complete, exiting puppeteer fetch.")

    async def main(self, **kwargs):
        await self.fetch_page(**kwargs)

    async def run(self,
                  fetch_favicon=True,
                  current_include_filters=None,
                  empty_pages_are_a_change=False,
                  ignore_status_codes=False,
                  is_binary=False,
                  request_body=None,
                  request_headers=None,

View on GitHub (pinned to 5d9c7c6da7)

Solutions

Switch the watch to the Playwright fetcher (scraper maintained there)
Retry — often a navigation race
Report the URL upstream if it reproduces on current versions

Defensive patterns

Strategy: fallback

Try / catch

try:
    fetcher.fetch_page()
except Exception as e:
    if 'xPath scraper failed' in str(e):
        switch_watch_fetcher(watch, 'playwright')
        retry_check()

Prevention

Use playwright fetcher when visual-selector data is needed
Retry once on scraper failure — often a race
Report reproducible URLs upstream

When it happens

Trigger: Running the puppeteer fetcher with visual-selector xpath scraping where evaluate() returns None/empty — page context destroyed, navigation race, or protocol incompatibility.

Common situations: pyppeteer/Chrome protocol drift making evaluate return nothing; page navigating during scrape; extension/csp interference.

Related errors

AI-assisted analysis of dgtlmoon/changedetection.io@5d9c7c6da7 (2026-08-27). Data as JSON: /api/errors/4fbf7870abf8c871. Report an issue: GitHub.