{"record":{"id":"f7381c4f1bb90788","repo":"D4Vinci/Scrapling","slug":"checkpoints-interval-must-be-equal-or-greater-than","errorCode":null,"errorMessage":"Checkpoints interval must be equal or greater than 0.","messagePattern":"Checkpoints interval must be equal or greater than 0\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scrapling/spiders/checkpoint.py","lineNumber":36,"sourceCode":"\n    requests: List[\"Request\"] = field(default_factory=list)\n    seen: Set[bytes] = field(default_factory=set)\n\n\nclass CheckpointManager:\n    \"\"\"Manages saving and loading checkpoint state to/from disk.\"\"\"\n\n    CHECKPOINT_FILE = \"checkpoint.pkl\"\n\n    def __init__(self, crawldir: str | Path | AsyncPath, interval: float = 300.0):\n        self.crawldir = AsyncPath(crawldir)\n        self._checkpoint_path = self.crawldir / self.CHECKPOINT_FILE\n        self.interval = interval\n        if not isinstance(interval, (int, float)):\n            raise TypeError(\"Checkpoints interval must be integer or float.\")\n        else:\n            if interval < 0:\n                raise ValueError(\"Checkpoints interval must be equal or greater than 0.\")\n\n    async def has_checkpoint(self) -> bool:\n        \"\"\"Check if a checkpoint exists.\"\"\"\n        return await self._checkpoint_path.exists()\n\n    async def save(self, data: CheckpointData) -> None:\n        \"\"\"Save checkpoint data to disk atomically.\"\"\"\n        await self.crawldir.mkdir(parents=True, exist_ok=True)\n\n        temp_path = self._checkpoint_path.with_suffix(\".tmp\")\n\n        try:\n            serialized = pickle.dumps(data, protocol=pickle.HIGHEST_PROTOCOL)\n            async with await anyio.open_file(temp_path, \"wb\") as f:\n                await f.write(serialized)\n\n            await temp_path.replace(self._checkpoint_path)\n","sourceCodeStart":18,"sourceCodeEnd":54,"githubUrl":"https://github.com/D4Vinci/Scrapling/blob/5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f/scrapling/spiders/checkpoint.py#L18-L54","documentation":"CheckpointManager rejects negative intervals: a checkpoint every N seconds makes no sense for N < 0, so __init__ raises ValueError. Zero is allowed (checkpoint on every save opportunity), as is any positive int/float.","triggerScenarios":"CheckpointManager(crawldir, interval=-300) or interval=-0.5, often from arithmetic that produced a negative number (e.g. subtracting a larger configured delay from a budget).","commonSituations":"Computing interval as a difference (deadline - elapsed) that went negative; a sentinel value like -1 meaning 'disabled' leaking from config into the manager.","solutions":["Clamp the computed value: interval=max(0, computed_interval).","If -1 means 'disable checkpoints' in your config, branch and skip creating the manager or use interval=0 deliberately.","Audit the arithmetic that produced the interval — a negative value usually indicates a deadline already passed."],"exampleFix":"# before\nCheckpointManager(crawldir, interval=remaining_seconds)  # ValueError if negative\n\n# after\nCheckpointManager(crawldir, interval=max(0.0, remaining_seconds))","handlingStrategy":"validation","validationCode":"interval = float(cfg.get('interval', 300))\ninterval = max(0.0, interval)  # zero is allowed, negatives are not\nCheckpointManager(crawldir, interval=interval)","typeGuard":"def is_non_negative_number(value) -> bool:\n    return isinstance(value, (int, float)) and not isinstance(value, bool) and value >= 0","tryCatchPattern":null,"preventionTips":["Clamp computed intervals with max(0, x).","Do not use -1 as a 'disabled' sentinel — translate sentinels to 0 or skip manager creation."],"tags":["checkpoint","validation","value-error","configuration"],"backgroundTag":null,"analyzedSha":"5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f","analyzedAt":"2026-08-14T22:23:09.440Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}