What causes "A system-error occurred when validating your XPath expression" in dgtlmoon/changedetection.io?

Reproduce outside the app: build an empty tree and call elementpath.select(tree, expr, parser=SafeXPath3Parser) to see the true exception

dgtlmoon/changedetection.io · error · ValidationError

A system-error occurred when validating your XPath expressio

Error message

A system-error occurred when validating your XPath expression

What it means

This is the catch-all except: clause in the XPath validation block. If elementpath.select or the lxml tree construction raises anything other than elementpath.ElementPathError (e.g. XMLSyntaxError from lxml, TypeError, MemoryError, or a parser crash), the validator hides the details and raises the generic "A system-error occurred when validating your XPath expression" ValidationError.

Source

Thrown at changedetectionio/forms.py:711

            # Does it look like XPath?
            if line.strip()[0] == '/' or line.strip().startswith('xpath:'):
                if not self.allow_xpath:
                    raise ValidationError("XPath not permitted in this field!")
                from lxml import etree, html
                import elementpath
                from changedetectionio.html_tools import SafeXPath3Parser, lxml_guard, lxml_html_parser
                line = line.replace('xpath:', '')

                try:
                    # Runs on a Flask request thread - must share the worker's lxml lock.
                    with lxml_guard():
                        tree = html.fromstring("<html></html>", parser=lxml_html_parser())
                        elementpath.select(tree, line.strip(), parser=SafeXPath3Parser)
                except elementpath.ElementPathError as e:
                    message = field.gettext('\'%(expression)s\' is not a valid XPath expression. (%(error)s)')
                    raise ValidationError(message % {'expression': line, 'error': str(e)})
                except:
                    raise ValidationError("A system-error occurred when validating your XPath expression")

            if line.strip().startswith('xpath1:'):
                if not self.allow_xpath:
                    raise ValidationError("XPath not permitted in this field!")
                from lxml import etree, html
                from changedetectionio.html_tools import lxml_guard, lxml_html_parser
                line = re.sub(r'^xpath1:', '', line)

                try:
                    # Runs on a Flask request thread - must share the worker's lxml lock.
                    with lxml_guard():
                        tree = html.fromstring("<html></html>", parser=lxml_html_parser())
                        tree.xpath(line.strip())
                except etree.XPathEvalError as e:
                    message = field.gettext('\'%(expression)s\' is not a valid XPath expression. (%(error)s)')
                    raise ValidationError(message % {'expression': line, 'error': str(e)})
                except:
                    raise ValidationError("A system-error occurred when validating your XPath expression")

View on GitHub (pinned to 5d9c7c6da7)

Solutions

Reproduce outside the app: build an empty tree and call elementpath.select(tree, expr, parser=SafeXPath3Parser) to see the true exception
Pin/align lxml and elementpath versions to known-good releases used by changedetection.io
Simplify the XPath expression to standard constructs; if it errors standalone, report the underlying exception upstream
Check the server logs/interactive debugger for the suppressed original exception

Example fix

# repro to find the real error
from lxml import html
import elementpath
from changedetectionio.html_tools import SafeXPath3Parser
tree = html.fromstring('<html></html>')
elementpath.select(tree, "//div[text()='x']", parser=SafeXPath3Parser)

Defensive patterns

Strategy: try-catch

Validate before calling

def xpath_safe_check(expr: str) -> str | None:
    try:
        from lxml import html
        import elementpath
        from changedetectionio.html_tools import SafeXPath3Parser
        tree = html.fromstring('<html></html>')
        elementpath.select(tree, expr, parser=SafeXPath3Parser)
        return None
    except Exception as e:  # surface the real error
        return repr(e)

Try / catch

try:
    elementpath.select(tree, expr, parser=SafeXPath3Parser)
except elementpath.ElementPathError as e:
    flash(f'Invalid XPath: {e}')
except Exception as e:
    log.exception('XPath validator crashed')  # keep the real traceback
    flash('System error validating XPath — check server logs')

Prevention

Pin compatible lxml/elementpath versions
Simplify exotic XPath constructs
Log the original exception; never rely on the bare-except message for diagnosis

When it happens

Trigger: An XPath expression that passes initial parsing but crashes lxml/elementpath at evaluation time on the empty <html></html> tree — for example expressions with unusual node tests, huge/recursive structures, or library-level bugs/version incompatibilities between lxml and elementpath.

Common situations: Upgrading lxml or elementpath to incompatible versions, using exotic XPath 3.0 constructs that hit unimplemented code paths, or environment issues (broken libxml2). The bare except also masks the real traceback, making diagnosis hard.

Related errors

AI-assisted analysis of dgtlmoon/changedetection.io@5d9c7c6da7 (2026-08-27). Data as JSON: /api/errors/649d39fb6063503f. Report an issue: GitHub.