{"record":{"id":"ad99e87d09c71e57","repo":"D4Vinci/Scrapling","slug":"cannot-compare-requests-before-generating-their-fi","errorCode":null,"errorMessage":"Cannot compare requests before generating their fingerprints!","messagePattern":"Cannot compare requests before generating their fingerprints!","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"scrapling/spiders/request.py","lineNumber":150,"sourceCode":"\n    def __lt__(self, other: object) -> bool:\n        \"\"\"Compare requests by priority\"\"\"\n        if not isinstance(other, Request):\n            return NotImplemented\n        return self.priority < other.priority\n\n    def __gt__(self, other: object) -> bool:\n        \"\"\"Compare requests by priority\"\"\"\n        if not isinstance(other, Request):\n            return NotImplemented\n        return self.priority > other.priority\n\n    def __eq__(self, other: object) -> bool:\n        \"\"\"Requests are equal if they have the same fingerprint.\"\"\"\n        if not isinstance(other, Request):\n            return NotImplemented\n        if self._fp is None or other._fp is None:\n            raise RuntimeError(\"Cannot compare requests before generating their fingerprints!\")\n        return self._fp == other._fp\n\n    def __getstate__(self) -> dict[str, Any]:\n        \"\"\"Prepare state for pickling - store callback as name string for pickle compatibility.\"\"\"\n        state = self.__dict__.copy()\n        state[\"_callback_name\"] = getattr(self.callback, \"__name__\", None) if self.callback is not None else None\n        state[\"callback\"] = None  # Don't pickle the actual callable\n        return state\n\n    def __setstate__(self, state: dict[str, Any]) -> None:\n        \"\"\"Restore state from pickle - callback restored later via _restore_callback().\"\"\"\n        self._callback_name: str | None = state.pop(\"_callback_name\", None)\n        self.__dict__.update(state)\n\n    def _restore_callback(self, spider: \"Spider\") -> None:\n        \"\"\"Restore callback from spider after unpickling.\n\n        :param spider: Spider instance to look up callback method on","sourceCodeStart":132,"sourceCodeEnd":168,"githubUrl":"https://github.com/D4Vinci/Scrapling/blob/5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f/scrapling/spiders/request.py#L132-L168","documentation":"Request.__eq__ compares requests by fingerprint, but the fingerprint (_fp) is computed lazily. If two Request objects are compared with == (or used in a set / as dict keys, which invokes __hash__ via the same state) before the fingerprint was ever generated, the comparison is impossible and raises RuntimeError.","triggerScenarios":"Creating two Request objects and immediately checking req_a == req_b, or putting fresh Requests into a set(), without touching req.fingerprint (or a scheduler step) first.","commonSituations":"Writing unit tests for dedup logic with hand-built Request objects; grouping requests manually before handing them to the spider scheduler that would normally generate fingerprints.","solutions":["Generate the fingerprint explicitly before comparing: _ = req_a.fingerprint; _ = req_b.fingerprint (or whatever the documented accessor/trigger is — accessing req.fingerprint populates _fp).","In dedup code, key on req.fingerprint directly instead of relying on __eq__/sets of Request objects.","If you control the flow, funnel requests through the framework's scheduling step first, which fingerprints them."],"exampleFix":"# before\nif req_a == req_b:  # RuntimeError if fingerprints not generated\n\n# after\nif req_a.fingerprint == req_b.fingerprint:  # accessors compute _fp lazily","handlingStrategy":"validation","validationCode":"# force lazy fingerprint generation before any comparison or set/dict use\n_ = req_a.fingerprint\n_ = req_b.fingerprint\n\nseen = {req.fingerprint for req in requests}  # key on fingerprint, not Request","typeGuard":"def has_fingerprint(req) -> bool:\n    return getattr(req, '_fp', None) is not None","tryCatchPattern":"try:\n    same = req_a == req_b\nexcept RuntimeError:\n    _ = req_a.fingerprint, req_b.fingerprint\n    same = req_a == req_b","preventionTips":["Key dedup structures on req.fingerprint instead of Request objects.","In tests, touch req.fingerprint once after construction before asserting equality."],"tags":["request","fingerprint","comparison","lazy-initialization"],"backgroundTag":null,"analyzedSha":"5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f","analyzedAt":"2026-08-14T22:23:09.440Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}