{"record":{"id":"72c216237df95a1a","repo":"D4Vinci/Scrapling","slug":"spider-has-no-starting-point-either-set-start-ur","errorCode":null,"errorMessage":"Spider has no starting point, either set `start_urls` or override `start_requests` function.","messagePattern":"Spider has no starting point, either set `start_urls` or override `start_requests` function\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"scrapling/spiders/spider.py","lineNumber":165,"sourceCode":"        except Exception as e:\n            raise SessionConfigurationError(f\"Error in {self.__class__.__name__}.configure_sessions(): {e}\") from e\n\n        if len(self._session_manager) == 0:\n            raise SessionConfigurationError(f\"{self.__class__.__name__}.configure_sessions() did not add any sessions\")\n\n        self.logger.info(\"Spider initialized\")\n\n    async def start_requests(self) -> AsyncGenerator[Request, None]:\n        \"\"\"Generate initial requests to start the crawl.\n\n        By default, this generates Request objects for each URL in `start_urls`\n        using the session manager's default session and `parse()` as callback.\n\n        Override this method for more control over initial requests\n        (e.g., to add custom headers, use different callbacks, etc.)\n        \"\"\"\n        if not self.start_urls:\n            raise RuntimeError(\n                \"Spider has no starting point, either set `start_urls` or override `start_requests` function.\"\n            )\n\n        for url in self.start_urls:\n            yield Request(url, sid=self._session_manager.default_session_id)\n\n    @abstractmethod\n    async def parse(self, response: \"Response\") -> AsyncGenerator[Dict[str, Any] | Request | None, None]:\n        \"\"\"Default callback for processing responses\"\"\"\n        raise NotImplementedError(f\"{self.__class__.__name__} must implement parse() method\")\n        yield  # Make this a generator for type checkers\n\n    async def on_start(self, resuming: bool = False) -> None:\n        \"\"\"Called before crawling starts. Override for setup logic.\n\n        :param resuming: It's enabled if the spider is resuming from a checkpoint, left for the user to use.\n        \"\"\"\n        if resuming:","sourceCodeStart":147,"sourceCodeEnd":183,"githubUrl":"https://github.com/D4Vinci/Scrapling/blob/5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f/scrapling/spiders/spider.py#L147-L183","documentation":"The default Spider.start_requests() requires at least one entry in the class attribute start_urls. If it's empty/None and start_requests isn't overridden, there is nothing to crawl and the engine raises at crawl start.","triggerScenarios":"Defining a spider with neither start_urls nor a start_requests override; setting start_urls = [] (empty list) expecting requests to come from elsewhere; overriding __init__ and overwriting start_urls with an empty value.","commonSituations":"Spiders that intend to generate requests dynamically but forgot to override start_requests; config-driven spiders where the URL list came back empty; renaming the attribute (e.g. start_url singular).","solutions":["Set start_urls = [\"https://example.com\"] as a class attribute","Or override `async def start_requests(self)` and yield Request(url, sid=self._session_manager.default_session_id) yourself","Verify the attribute is exactly `start_urls` (plural) and non-empty at runtime"],"exampleFix":"// before\nclass MySpider(Spider):\n    name = \"my\"\n    async def parse(self, response): ...\n\n// after\nclass MySpider(Spider):\n    name = \"my\"\n    start_urls = [\"https://example.com\"]\n    async def parse(self, response): ...","handlingStrategy":"validation","validationCode":"if not getattr(MySpider, \"start_urls\", None) and Spider.start_requests is MySpider.start_requests:\n    raise SystemExit(\"set start_urls or override start_requests\")","typeGuard":"def has_start_point(spider_cls) -> bool:\n    return bool(getattr(spider_cls, \"start_urls\", None)) or spider_cls.start_requests is not Spider.start_requests","tryCatchPattern":null,"preventionTips":["Spell the attribute exactly start_urls (plural)","Override start_requests for dynamic entry points","Assert a start point in a smoke test for each spider"],"tags":["spider","configuration","validation"],"backgroundTag":null,"analyzedSha":"5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f","analyzedAt":"2026-08-14T22:23:09.440Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}