{"record":{"id":"fec47366b2fb7e28","repo":"D4Vinci/Scrapling","slug":"storage-system-must-implement-retrieve-method","errorCode":null,"errorMessage":"Storage system must implement `retrieve` method","messagePattern":"Storage system must implement `retrieve` method","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"scrapling/core/storage.py","lineNumber":59,"sourceCode":"    @abstractmethod\n    def save(self, element: HtmlElement, identifier: str) -> None:\n        \"\"\"Saves the element's unique properties to the storage for retrieval and relocation later\n\n        :param element: The element itself which we want to save to storage.\n        :param identifier: This is the identifier that will be used to retrieve the element later from the storage. See\n            the docs for more info.\n        \"\"\"\n        raise NotImplementedError(\"Storage system must implement `save` method\")\n\n    @abstractmethod\n    def retrieve(self, identifier: str) -> Optional[Dict]:\n        \"\"\"Using the identifier, we search the storage and return the unique properties of the element\n\n        :param identifier: This is the identifier that will be used to retrieve the element from the storage. See\n            the docs for more info.\n        :return: A dictionary of the unique properties\n        \"\"\"\n        raise NotImplementedError(\"Storage system must implement `retrieve` method\")\n\n    @staticmethod\n    @lru_cache(128, typed=True)\n    def _get_hash(identifier: str) -> str:\n        \"\"\"If you want to hash identifier in your storage system, use this safer\"\"\"\n        _identifier = identifier.lower().strip()\n        # Hash functions have to take bytes\n        _identifier_bytes = _identifier.encode(\"utf-8\")\n\n        hash_value = sha256(_identifier_bytes).hexdigest()\n        return f\"{hash_value}_{len(_identifier_bytes)}\"  # Length to reduce collision chance\n\n\n@lru_cache(1, typed=True)\nclass SQLiteStorageSystem(StorageSystemMixin):\n    \"\"\"The recommended system to use, it's race condition safe and thread safe.\n    Mainly built, so the library can run in threaded frameworks like scrapy or threaded tools\n    > It's optimized for threaded applications, but running it without threads shouldn't make it slow.\"\"\"","sourceCodeStart":41,"sourceCodeEnd":77,"githubUrl":"https://github.com/D4Vinci/Scrapling/blob/5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f/scrapling/core/storage.py#L41-L77","documentation":"Raised by the abstract `retrieve` method of `StorageSystemMixin` in scrapling/core/storage.py. `retrieve(identifier)` is what adaptive fetching calls to look up an element's previously saved properties (via `element.retrieve()` / auto-retrieve). Like its `save` sibling, the ABC raises `NotImplementedError` when a custom storage backend fails to override it. The bundled `SQLiteStorageSystem` implements lookup against its SQLite file, so this error only appears with user-defined storage classes.","triggerScenarios":"A custom `StorageSystemMixin` subclass that implements `save` but not `retrieve`, followed by any adaptive retrieval call: `page.css('h1').relocate()` or constructing a Selector with `adaptive=True` / `storage=MyStorage()` so retrieval looks up a saved identifier.","commonSituations":"Porting a storage backend from an older Scrapling version where the interface had fewer methods; implementing only the write path during development and forgetting the read path; typo'd method name such as `get` or `load` instead of `retrieve`.","solutions":["Implement `def retrieve(self, identifier: str) -> Optional[Dict]` in your subclass, returning the saved property dict or `None` when not found.","Mirror the reference implementation in `SQLiteStorageSystem.retrieve` for the expected dict shape (`_StorageTools.element_to_dict` output).","Guard with `inspect.isabstract()` at import time so a half-finished backend fails fast.","Use the built-in SQLite backend if you don't actually need a custom store."],"exampleFix":"// before\nclass MyStorage(StorageSystemMixin):\n    def save(self, element, identifier): ...\n    # retrieve missing\n\n// after\nclass MyStorage(StorageSystemMixin):\n    def save(self, element, identifier): ...\n\n    def retrieve(self, identifier) -> Optional[Dict]:\n        row = self._db.get(self._get_hash(identifier))\n        return row  # dict of unique properties, or None","handlingStrategy":"validation","validationCode":"def assert_storage_retrieve(storage) -> None:\n    import inspect\n    m = type(storage).retrieve\n    assert getattr(m, '__isabstractmethod__', False) is False or storage.__class__ is not None, 'retrieve not overridden'\n    # strongest check: no abstract methods left on the concrete class\n    assert not type(storage).__abstractmethods__, f\"abstract methods left: {type(storage).__abstractmethods__}\"","typeGuard":"def is_concrete_storage(s) -> bool:\n    cls = type(s)\n    return not getattr(cls, '__abstractmethods__', None) and callable(getattr(cls, 'retrieve', None)) and not getattr(cls.retrieve, '__isabstractmethod__', False)","tryCatchPattern":"try:\n    props = element.retrieve(identifier='x')\nexcept NotImplementedError:\n    log.error('storage backend lacks retrieve(); implement it before adaptive lookup')\n    raise","preventionTips":["Implement both save and retrieve together — the interface is symmetric.","Return Optional[Dict]: None when not found instead of raising.","Unit-test retrieve with a saved identifier right after save in your backend's test suite."],"tags":["storage","abstract-method","custom-backend","scrapling"],"backgroundTag":null,"analyzedSha":"5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f","analyzedAt":"2026-08-14T22:23:09.440Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}