D4Vinci/Scrapling · error · NotImplementedError

Storage system must implement `save` method

Error message

Storage system must implement `save` method

What it means

Raised by the abstract `save` method of `StorageSystemMixin` (scrapling/core/storage.py), the base class for Scrapling's adaptive element-relocation storage backends. It fires when a custom storage class is instantiated without overriding `save`, because the ABC only enforces the method's existence, not its body. The bundled `SQLiteStorageSystem` implements it, so hitting this means you subclassed `StorageSystemMixin` yourself and left `save` abstract. The `raise NotImplementedError` inside an `@abstractmethod` body is the fallback that executes if instantiation slipped through (e.g. metaclass not applied).

Source

Thrown at scrapling/core/storage.py:49

            # Fixing the inaccurate return type hint in `get_tld`
            extracted: Result | None = cast(
                Result, get_tld(self.url, as_object=True, fail_silently=True, fix_protocol=True)
            )
            if not extracted:
                return default_value
            return extracted.fld or extracted.domain or default_value
        except AttributeError:
            return default_value

    @abstractmethod
    def save(self, element: HtmlElement, identifier: str) -> None:
        """Saves the element's unique properties to the storage for retrieval and relocation later

        :param element: The element itself which we want to save to storage.
        :param identifier: This is the identifier that will be used to retrieve the element later from the storage. See
            the docs for more info.
        """
        raise NotImplementedError("Storage system must implement `save` method")

    @abstractmethod
    def retrieve(self, identifier: str) -> Optional[Dict]:
        """Using the identifier, we search the storage and return the unique properties of the element

        :param identifier: This is the identifier that will be used to retrieve the element from the storage. See
            the docs for more info.
        :return: A dictionary of the unique properties
        """
        raise NotImplementedError("Storage system must implement `retrieve` method")

    @staticmethod
    @lru_cache(128, typed=True)
    def _get_hash(identifier: str) -> str:
        """If you want to hash identifier in your storage system, use this safer"""
        _identifier = identifier.lower().strip()
        # Hash functions have to take bytes
        _identifier_bytes = _identifier.encode("utf-8")

View on GitHub (pinned to 5d213a2d47)

Solutions

1. Implement `def save(self, element: HtmlElement, identifier: str) -> None` in your storage subclass, persisting the element's identifying properties keyed by `identifier`.
2. If you wanted the default behavior, use the shipped `SQLiteStorageSystem` instead of subclassing `StorageSystemMixin` directly.
3. Verify the subclass is fully concrete before use with `import inspect; assert not inspect.isabstract(MyStorage)`.
4. If wrapping the built-in storage, delegate explicitly (`SQLiteStorageSystem().save(...)`) instead of relying on the abstract base body.

Example fix

// before
class RedisStorage(StorageSystemMixin):
    def retrieve(self, identifier):
        return redis.get(identifier)

// after
class RedisStorage(StorageSystemMixin):
    def save(self, element, identifier) -> None:
        data = _StorageTools.element_to_dict(element)
        redis.set(self._get_hash(identifier), json.dumps(data))

    def retrieve(self, identifier):
        return json.loads(redis.get(self._get_hash(identifier)))

import inspect

def make_storage(cls):
    missing = inspect.getabstractmethods(cls) if hasattr(cls, '__abstractmethods__') else ()
    # per-instance check
    if getattr(MyStorage, '__abstractmethods__', None):
        raise TypeError(f"Storage backend incomplete: {MyStorage.__abstractmethods__}")
    return cls()

from scrapling.core.storage import StorageSystemMixin

def is_complete_storage(s) -> bool:
    return isinstance(s, StorageSystemMixin) and not type(s).__abstractmethods__ and hasattr(type(s).save, '__isabstractmethod__') is False and not getattr(type(s).save, '__isabstractmethod__', True)

try:
    element.save(identifier='x')
except NotImplementedError as e:
    log.error(f"Storage backend misconfigured: {e}")  # fix the subclass, don't retry

When it happens

Trigger: Defining `class MyStorage(StorageSystemMixin)` with `retrieve` implemented but `save` missing or named differently (e.g. `save_element`, typo `savr`), then passing an instance to a Selector/adaptive feature that calls `storage.save(element, identifier)` during `element.save()` or auto-save flows.

Common situations: Writing a Redis/Mongo storage backend (as shown in docs/development/adaptive_storage_system.md) and forgetting one of the two required methods; renaming methods during a refactor; subclassing `SQLiteStorageSystem` and accidentally overriding `save` with a stub that calls `super().save()`.

Related errors

• Storage system must implement `retrieve` method
• Expressions of type {__name__}.XPathExpr can ony join expres
• The functional pseudo-element ::{pseudo_element.name}() is u
• The pseudo-element ::{pseudo_element} is unknown
• Expected a single string or ident for ::attr(), got {functio

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/4c376915bd20cc97. Report an issue: GitHub.