D4Vinci/Scrapling · error · SelectorSyntaxError

Invalid XPath selector: {selector}

Error message

Invalid XPath selector: {selector}

What it means

The xpath() method failed to compile or evaluate the XPath expression. lxml raised XPathError/XPathEvalError (or cssselect raised a SelectorError) and scrapling re-raises it as SelectorSyntaxError with the original exception attached as __cause__. Note the message does not echo the underlying reason, so inspect __cause__ for details.

Source

Thrown at scrapling/parser.py:694

            else:
                if adaptive:
                    log.warning(
                        "Argument `adaptive` will be ignored because `adaptive` wasn't enabled on initialization. Check docs for more info."
                    )
                elif auto_save:
                    log.warning(
                        "Argument `auto_save` will be ignored because `adaptive` wasn't enabled on initialization. Check docs for more info."
                    )

                return self.__handle_elements(elements)

        except (
            SelectorError,
            SelectorSyntaxError,
            XPathError,
            XPathEvalError,
        ) as e:
            raise SelectorSyntaxError(f"Invalid XPath selector: {selector}") from e

    def find_all(
        self,
        *args: str | Iterable[str] | Pattern | Callable | Dict[str, str],
        **kwargs: str,
    ) -> "Selectors":
        """Find elements by filters of your creations for ease.

        :param args: Tag name(s), iterable of tag names, regex patterns, function, or a dictionary of elements' attributes. Leave empty for selecting all.
        :param kwargs: The attributes you want to filter elements based on it.
        :return: The `Selectors` object of the elements or empty list
        """
        if self._is_text_node(self._root):
            return Selectors()

        if not args and not kwargs:
            raise TypeError("You have to pass something to search with, like tag name(s), tag attributes, or both.")

View on GitHub (pinned to 5d213a2d47)

Solutions

1. Inspect e.__cause__ — the lxml XPathEvalError message names the unsupported function or syntax error.
2. Replace XPath 2.0 constructs: ends-with(x, y) -> substring(x, string-length(x) - string-length(y) + 1) = y; matches() -> re:test() from lxml's EXSLT extension.
3. Fix quoting: if the expression contains double quotes, wrap literals in single quotes or escape as ".
4. Register lxml extension namespaces (xmlns:re) when using EXSLT regex functions.

Example fix

# before
page.xpath("//div[ends-with(@id, '-main')]")  # SelectorSyntaxError

# after
page.xpath("//div[substring(@id, string-length(@id) - 4) = '-main']")

from lxml import etree

def xpath_is_valid(expr: str) -> bool:
    try:
        etree.XPath(expr)
        return True
    except etree.XPathError:
        return False

from scrapling.core.exceptions import SelectorSyntaxError

try:
    nodes = page.xpath(expr)
except SelectorSyntaxError as e:
    cause = e.__cause__  # lxml XPathError has the real reason
    logger.error('bad xpath %r: %s', expr, cause)
    nodes = page.xpath('.//*')  # fallback or re-raise

When it happens

Trigger: Calling selector.xpath('<bad expr>') with invalid syntax like '//div[[', unsupported XPath 2.0+ functions (ends-with, matches, string-join), or expressions that return a non-node result in a context expecting nodes.

Common situations: Using XPath 2.0 functions that lxml's XPath 1.0 engine doesn't implement; unescaped quotes inside string literals of dynamically built expressions; expressions copied from XPath 2.0/3.0 tooling.

Related errors

• Selector class needs HTML content, or root arguments to work
• content argument must be str or bytes, got {type(content)}
• Text nodes do not have attributes
• Invalid CSS selector '{selector}': {str(e)}
• 'quality' is only valid when 'image_type' is 'jpeg'.

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/1a33e4eef6ba9130. Report an issue: GitHub.