D4Vinci/Scrapling · error · ValueError

Storage system must be inherited from class `StorageSystemMi

Error message

Storage system must be inherited from class `StorageSystemMixin`

What it means

ValueError raised after the lru_cache check: scrapling unwraps the decorated `storage` argument via `storage.__wrapped__` and verifies it is a subclass of `StorageSystemMixin`. If the wrapped callable is some other class (arbitrary lru_cache-wrapped class/function), the storage system does not implement the interface Selector needs and is rejected. Marked `# pragma: no cover` — an internal invariant in normal use.

Source

Thrown at scrapling/parser.py:179

                return

        self.__adaptive_enabled = bool(adaptive)

        if self.__adaptive_enabled:
            if _storage is not None:
                self._storage = _storage
            else:
                if not storage_args:
                    storage_args = {
                        "storage_file": __DEFAULT_DB_FILE__,
                        "url": url,
                    }

                if not hasattr(storage, "__wrapped__"):
                    raise ValueError("Storage class must be wrapped with lru_cache decorator, see docs for info")

                if not issubclass(storage.__wrapped__, StorageSystemMixin):  # pragma: no cover
                    raise ValueError("Storage system must be inherited from class `StorageSystemMixin`")

                self._storage = storage(**storage_args)

    def __getitem__(self, key: str) -> TextHandler:
        if self._is_text_node(self._root):
            raise TypeError("Text nodes do not have attributes")
        return self.attrib[key]

    def __contains__(self, key: str) -> bool:
        if self._is_text_node(self._root):
            return False
        return key in self.attrib

    # Node functionalities, I wanted to move to a separate Mixin class, but it had a slight impact on performance
    @staticmethod
    def _is_text_node(
        element: HtmlElement | _ElementUnicodeResult,
    ) -> bool:

View on GitHub (pinned to 5d213a2d47)

Solutions

1. Inherit from scrapling's StorageSystemMixin: class MyStorage(StorageSystemMixin): ... and keep the @lru_cache wrapper.
2. Implement the required save/load interface defined by StorageSystemMixin on your subclass.
3. If you do not need custom persistence, drop the storage argument and use the default storage.

Example fix

# before
from functools import lru_cache
@lru_cache
class MyStorage:  # no mixin
    ...

# after
from functools import lru_cache
from scrapling.core.storage import StorageSystemMixin
@lru_cache
class MyStorage(StorageSystemMixin):
    ...

from scrapling.core.storage import StorageSystemMixin

def is_storage_subclass(storage) -> bool:
    inner = getattr(storage, "__wrapped__", None)
    return isinstance(inner, type) and issubclass(inner, StorageSystemMixin)

assert is_storage_subclass(storage), "storage must subclass StorageSystemMixin (and be @lru_cache wrapped)"

def is_valid_storage(storage: object) -> bool:
    inner = getattr(storage, "__wrapped__", None)
    return isinstance(inner, type) and issubclass(inner, StorageSystemMixin)

try:
    sel = Selector(content=html, adaptive=True, storage=storage)
except ValueError as e:
    if "StorageSystemMixin" in str(e):
        raise ConfigError("custom storage must inherit StorageSystemMixin") from e
    raise

When it happens

Trigger: Passing an lru_cache-wrapped class that does not inherit from scrapling's StorageSystemMixin, e.g. @lru_cache class MyDB: ... used with Selector(..., adaptive=True, storage=MyDB); or passing the decorator itself rather than a storage class.

Common situations: Adapting a third-party storage/key-value class and forgetting the mixin; following the lru_cache requirement but skipping the inheritance requirement from the docs.

Related errors

• Storage class must be wrapped with lru_cache decorator, see
• Unknown parser argument: "{key}"; maybe you meant {cls.parse
• You must pass a keyword to configure, current keywords: {cls
• Selector class needs HTML content, or root arguments to work
• content argument must be str or bytes, got {type(content)}

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/bac2f9af8aeb29f4. Report an issue: GitHub.