{"record":{"id":"4fbf7870abf8c871","repo":"dgtlmoon/changedetection.io","slug":"content-fetcher-xpath-scraper-failed-please-rep","errorCode":null,"errorMessage":"Content Fetcher > xPath scraper failed. Please report this URL so we can fix it :)","messagePattern":"Content Fetcher > xPath scraper failed\\. Please report this URL so we can fix it :\\)","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"changedetectionio/content_fetchers/puppeteer.py","lineNumber":498,"sourceCode":"            await self.page.evaluate(f\"var include_filters=''\")\n\n        MAX_TOTAL_HEIGHT = int(os.getenv(\"SCREENSHOT_MAX_HEIGHT\", SCREENSHOT_MAX_HEIGHT_DEFAULT))\n\n        self.content = await self.page.content\n\n        # Now take screenshot (scrolling may trigger layout changes, but measurements are already captured)\n        logger.debug(f\"Screenshot format {self.screenshot_format}\")\n        self.screenshot = await capture_full_page(page=self.page, screenshot_format=self.screenshot_format, watch_uuid=watch_uuid, lock_viewport_elements=self.lock_viewport_elements)\n\n        # Force garbage collection - pyppeteer base64 decode creates temporary buffers\n        import gc\n        gc.collect()\n        self.xpath_data = await self.page.evaluate(XPATH_ELEMENT_JS, {\n            \"visualselector_xpath_selectors\": visualselector_xpath_selectors,\n            \"max_height\": MAX_TOTAL_HEIGHT\n        })\n        if not self.xpath_data:\n            raise Exception(f\"Content Fetcher > xPath scraper failed. Please report this URL so we can fix it :)\")\n\n\n        self.instock_data = await self.page.evaluate(INSTOCK_DATA_JS)\n\n        # It's good to log here in the case that the browser crashes on shutting down but we still get the data we need\n        logger.success(f\"Fetching '{url}' complete, exiting puppeteer fetch.\")\n\n    async def main(self, **kwargs):\n        await self.fetch_page(**kwargs)\n\n    async def run(self,\n                  fetch_favicon=True,\n                  current_include_filters=None,\n                  empty_pages_are_a_change=False,\n                  ignore_status_codes=False,\n                  is_binary=False,\n                  request_body=None,\n                  request_headers=None,","sourceCodeStart":480,"sourceCodeEnd":516,"githubUrl":"https://github.com/dgtlmoon/changedetection.io/blob/5d9c7c6da76340597243e8163c4f2439237fa0e8/changedetectionio/content_fetchers/puppeteer.py#L480-L516","documentation":"Generic Exception raised when page.evaluate of the xpath-element scraper JS returns falsy — the in-page xpath extraction script produced no data, treated as an internal scraper failure the user is asked to report.","triggerScenarios":"Running the puppeteer fetcher with visual-selector xpath scraping where evaluate() returns None/empty — page context destroyed, navigation race, or protocol incompatibility.","commonSituations":"pyppeteer/Chrome protocol drift making evaluate return nothing; page navigating during scrape; extension/csp interference.","solutions":["Switch the watch to the Playwright fetcher (scraper maintained there)","Retry — often a navigation race","Report the URL upstream if it reproduces on current versions"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    fetcher.fetch_page()\nexcept Exception as e:\n    if 'xPath scraper failed' in str(e):\n        switch_watch_fetcher(watch, 'playwright')\n        retry_check()","preventionTips":["Use playwright fetcher when visual-selector data is needed","Retry once on scraper failure — often a race","Report reproducible URLs upstream"],"tags":["puppeteer","xpath","scraper"],"backgroundTag":"scraper-returned-no-data","analyzedSha":"5d9c7c6da76340597243e8163c4f2439237fa0e8","analyzedAt":"2026-08-27T19:41:16.067Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}