{"record":{"id":"d3d8e51f908f2a64","repo":"D4Vinci/Scrapling","slug":"can-t-pickle-selectors-object","errorCode":null,"errorMessage":"Can't pickle Selectors object","messagePattern":"Can't pickle Selectors object","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"scrapling/parser.py","lineNumber":1376,"sourceCode":"\n    @property\n    def first(self) -> Optional[Selector]:\n        \"\"\"Returns the first Selector item of the current list or `None` if the list is empty\"\"\"\n        return self[0] if len(self) > 0 else None\n\n    @property\n    def last(self) -> Optional[Selector]:\n        \"\"\"Returns the last Selector item of the current list or `None` if the list is empty\"\"\"\n        return self[-1] if len(self) > 0 else None\n\n    @property\n    def length(self) -> int:\n        \"\"\"Returns the length of the current list\"\"\"\n        return len(self)\n\n    def __getstate__(self) -> Any:  # pragma: no cover\n        # lxml don't like it :)\n        raise TypeError(\"Can't pickle Selectors object\")\n\n\n# For backward compatibility\nAdaptor = Selector\nAdaptors = Selectors\n","sourceCodeStart":1358,"sourceCodeEnd":1382,"githubUrl":"https://github.com/D4Vinci/Scrapling/blob/5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f/scrapling/parser.py#L1358-L1382","documentation":"Selectors (the list-like result container returned by css()/xpath()/find_all()) implements __getstate__ to raise TypeError on any pickling attempt. Each item wraps an lxml HtmlElement, which cannot be serialized, so copying or pickling the whole collection is blocked the same way as for a single Selector.","triggerScenarios":"pickle.dumps(page.css('a')), copy.deepcopy(results), multiprocessing.Pool.map over functions returning Selectors, or asyncio task result pickling.","commonSituations":"Parallelizing scraping across processes and returning matched element lists; memoizing query results; joblib caching of extraction functions.","solutions":["Convert to plain data before crossing a serialization boundary: [{'tag': s.tag, 'text': s.text, 'attrs': s.attrib} for s in results].","Use .html strings to carry element fragments across processes and re-parse on the other side.","Keep parsing inside one process; ship only URLs/HTML in, extracted primitives out."],"exampleFix":"# before\nresults = page.css('a')\npickle.dumps(results)  # TypeError\n\n# after\nresults = [{'text': s.text, 'href': s.attrib.get('href')} for s in page.css('a')]\npickle.dumps(results)","handlingStrategy":"validation","validationCode":"results = page.css('a')\npicklable = [s.html for s in results]          # str fragments\n# or full data:\npicklable = [{'tag': s.tag, 'text': s.text, 'attrib': dict(s.attrib)} for s in results]","typeGuard":"from scrapling.parser import Selectors\n\ndef is_selectors(obj) -> bool:\n    # true means: convert to primitives before pickling\n    return isinstance(obj, (Selectors, list)) and any(type(x).__name__ == 'Selector' for x in (obj if isinstance(obj, list) else list(obj)))","tryCatchPattern":"try:\n    pickle.dumps(payload)\nexcept TypeError as e:\n    if \"Can't pickle Selectors\" in str(e):\n        payload = [s.html for s in payload]\n    else:\n        raise","preventionTips":["Convert Selectors to str/dict immediately after querying when results must be cached or shipped.","Keep extraction functions (Selector in -> primitives out) as the boundary for multiprocessing workers."],"tags":["pickle","serialization","multiprocessing","selectors"],"backgroundTag":null,"analyzedSha":"5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f","analyzedAt":"2026-08-14T22:23:09.440Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}