{"record":{"id":"17e5254f571fedff","repo":"D4Vinci/Scrapling","slug":"checkpoints-interval-must-be-integer-or-float","errorCode":null,"errorMessage":"Checkpoints interval must be integer or float.","messagePattern":"Checkpoints interval must be integer or float\\.","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"scrapling/spiders/checkpoint.py","lineNumber":33,"sourceCode":"@dataclass\nclass CheckpointData:\n    \"\"\"Container for checkpoint state.\"\"\"\n\n    requests: List[\"Request\"] = field(default_factory=list)\n    seen: Set[bytes] = field(default_factory=set)\n\n\nclass CheckpointManager:\n    \"\"\"Manages saving and loading checkpoint state to/from disk.\"\"\"\n\n    CHECKPOINT_FILE = \"checkpoint.pkl\"\n\n    def __init__(self, crawldir: str | Path | AsyncPath, interval: float = 300.0):\n        self.crawldir = AsyncPath(crawldir)\n        self._checkpoint_path = self.crawldir / self.CHECKPOINT_FILE\n        self.interval = interval\n        if not isinstance(interval, (int, float)):\n            raise TypeError(\"Checkpoints interval must be integer or float.\")\n        else:\n            if interval < 0:\n                raise ValueError(\"Checkpoints interval must be equal or greater than 0.\")\n\n    async def has_checkpoint(self) -> bool:\n        \"\"\"Check if a checkpoint exists.\"\"\"\n        return await self._checkpoint_path.exists()\n\n    async def save(self, data: CheckpointData) -> None:\n        \"\"\"Save checkpoint data to disk atomically.\"\"\"\n        await self.crawldir.mkdir(parents=True, exist_ok=True)\n\n        temp_path = self._checkpoint_path.with_suffix(\".tmp\")\n\n        try:\n            serialized = pickle.dumps(data, protocol=pickle.HIGHEST_PROTOCOL)\n            async with await anyio.open_file(temp_path, \"wb\") as f:\n                await f.write(serialized)","sourceCodeStart":15,"sourceCodeEnd":51,"githubUrl":"https://github.com/D4Vinci/Scrapling/blob/5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f/scrapling/spiders/checkpoint.py#L15-L51","documentation":"CheckpointManager validates its checkpointing interval in __init__ and requires an int or float. Passing anything else (str, None, bool excluded types, Decimal) raises TypeError immediately at construction, before any crawling starts.","triggerScenarios":"CheckpointManager(crawldir, interval='300') — a string read from CLI args, environment variables, or a YAML/JSON config that wasn't cast to a number.","commonSituations":"Loading interval from settings: os.environ['INTERVAL'] or a config file yields str; passing a datetime.timedelta instead of seconds.","solutions":["Cast at the call site: CheckpointManager(crawldir, interval=float(cfg['interval'])).","If the value is a timedelta, use interval=td.total_seconds().","Validate config early with a schema/type check so bad values surface at startup with context."],"exampleFix":"# before\nCheckpointManager(crawldir, interval=os.environ['CHECKPOINT_INTERVAL'])  # TypeError\n\n# after\nCheckpointManager(crawldir, interval=float(os.environ['CHECKPOINT_INTERVAL']))","handlingStrategy":"validation","validationCode":"def to_interval_seconds(value) -> float:\n    if isinstance(value, str):\n        value = float(value)\n    if not isinstance(value, (int, float)) or isinstance(value, bool):\n        raise TypeError(f'interval must be int/float, got {type(value).__name__}')\n    return float(value)\n\nCheckpointManager(crawldir, interval=to_interval_seconds(cfg['interval']))","typeGuard":"def is_interval(value) -> bool:\n    return isinstance(value, (int, float)) and not isinstance(value, bool)","tryCatchPattern":null,"preventionTips":["Cast env/CLI/config values to float at load time.","Convert timedelta with .total_seconds() before passing."],"tags":["checkpoint","validation","type-error","configuration"],"backgroundTag":null,"analyzedSha":"5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f","analyzedAt":"2026-08-14T22:23:09.440Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}