{"record":{"id":"649d39fb6063503f","repo":"dgtlmoon/changedetection.io","slug":"a-system-error-occurred-when-validating-your-xpath","errorCode":null,"errorMessage":"A system-error occurred when validating your XPath expression","messagePattern":"A system-error occurred when validating your XPath expression","errorType":"validation","errorClass":"ValidationError","httpStatus":null,"severity":"error","filePath":"changedetectionio/forms.py","lineNumber":711,"sourceCode":" # Does it look like XPath?\n if line.strip()[0] == '/' or line.strip().startswith('xpath:'):\n if not self.allow_xpath:\n raise ValidationError(\"XPath not permitted in this field!\")\n from lxml import etree, html\n import elementpath\n from changedetectionio.html_tools import SafeXPath3Parser, lxml_guard, lxml_html_parser\n line = line.replace('xpath:', '')\n\n try:\n # Runs on a Flask request thread - must share the worker's lxml lock.\n with lxml_guard():\n tree = html.fromstring(\"\", parser=lxml_html_parser())\n elementpath.select(tree, line.strip(), parser=SafeXPath3Parser)\n except elementpath.ElementPathError as e:\n message = field.gettext('\\'%(expression)s\\' is not a valid XPath expression. (%(error)s)')\n raise ValidationError(message % {'expression': line, 'error': str(e)})\n except:\n raise ValidationError(\"A system-error occurred when validating your XPath expression\")\n\n if line.strip().startswith('xpath1:'):\n if not self.allow_xpath:\n raise ValidationError(\"XPath not permitted in this field!\")\n from lxml import etree, html\n from changedetectionio.html_tools import lxml_guard, lxml_html_parser\n line = re.sub(r'^xpath1:', '', line)\n\n try:\n # Runs on a Flask request thread - must share the worker's lxml lock.\n with lxml_guard():\n tree = html.fromstring(\"\", parser=lxml_html_parser())\n tree.xpath(line.strip())\n except etree.XPathEvalError as e:\n message = field.gettext('\\'%(expression)s\\' is not a valid XPath expression. (%(error)s)')\n raise ValidationError(message % {'expression': line, 'error': str(e)})\n except:\n raise ValidationError(\"A system-error occurred when validating your XPath expression\")","sourceCodeStart":693,"sourceCodeEnd":729,"githubUrl":"https://github.com/dgtlmoon/changedetection.io/blob/5d9c7c6da76340597243e8163c4f2439237fa0e8/changedetectionio/forms.py#L693-L729","documentation":"This is the catch-all except: clause in the XPath validation block. If elementpath.select or the lxml tree construction raises anything other than elementpath.ElementPathError (e.g. XMLSyntaxError from lxml, TypeError, MemoryError, or a parser crash), the validator hides the details and raises the generic \"A system-error occurred when validating your XPath expression\" ValidationError.","triggerScenarios":"An XPath expression that passes initial parsing but crashes lxml/elementpath at evaluation time on the empty tree — for example expressions with unusual node tests, huge/recursive structures, or library-level bugs/version incompatibilities between lxml and elementpath.","commonSituations":"Upgrading lxml or elementpath to incompatible versions, using exotic XPath 3.0 constructs that hit unimplemented code paths, or environment issues (broken libxml2). The bare except also masks the real traceback, making diagnosis hard.","solutions":["Reproduce outside the app: build an empty tree and call elementpath.select(tree, expr, parser=SafeXPath3Parser) to see the true exception","Pin/align lxml and elementpath versions to known-good releases used by changedetection.io","Simplify the XPath expression to standard constructs; if it errors standalone, report the underlying exception upstream","Check the server logs/interactive debugger for the suppressed original exception"],"exampleFix":"# repro to find the real error\nfrom lxml import html\nimport elementpath\nfrom changedetectionio.html_tools import SafeXPath3Parser\ntree = html.fromstring('')\nelementpath.select(tree, \"//div[text()='x']\", parser=SafeXPath3Parser)\n","handlingStrategy":"try-catch","validationCode":"def xpath_safe_check(expr: str) -> str | None:\n try:\n from lxml import html\n import elementpath\n from changedetectionio.html_tools import SafeXPath3Parser\n tree = html.fromstring('')\n elementpath.select(tree, expr, parser=SafeXPath3Parser)\n return None\n except Exception as e: # surface the real error\n return repr(e)\n","typeGuard":null,"tryCatchPattern":"try:\n elementpath.select(tree, expr, parser=SafeXPath3Parser)\nexcept elementpath.ElementPathError as e:\n flash(f'Invalid XPath: {e}')\nexcept Exception as e:\n log.exception('XPath validator crashed') # keep the real traceback\n flash('System error validating XPath — check server logs')\n","preventionTips":["Pin compatible lxml/elementpath versions","Simplify exotic XPath constructs","Log the original exception; never rely on the bare-except message for diagnosis"],"tags":["xpath","lxml","elementpath","catch-all","changedetectionio"],"backgroundTag":"xpath-evaluation-crash","analyzedSha":"5d9c7c6da76340597243e8163c4f2439237fa0e8","analyzedAt":"2026-08-27T19:41:16.067Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}