{"record":{"id":"34ea5d00a8d292e5","repo":"unclecode/crawl4ai","slug":"crawlerrunconfig-must-be-provided","errorCode":null,"errorMessage":"CrawlerRunConfig must be provided","messagePattern":"CrawlerRunConfig must be provided","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"crawl4ai/deep_crawling/base_strategy.py","lineNumber":100,"sourceCode":"    async def arun(\n        self,\n        start_url: str,\n        crawler: AsyncWebCrawler,\n        config: Optional[CrawlerRunConfig] = None,\n    ) -> RunManyReturn:\n        \"\"\"\n        Traverse the given URL using the specified crawler.\n        \n        Args:\n            start_url (str): The URL from which to start crawling.\n            crawler (AsyncWebCrawler): The crawler instance to use.\n            crawler_run_config (Optional[CrawlerRunConfig]): Crawler configuration.\n        \n        Returns:\n            Union[CrawlResultT, List[CrawlResultT], AsyncGenerator[CrawlResultT, None]]\n        \"\"\"\n        if config is None:\n            raise ValueError(\"CrawlerRunConfig must be provided\")\n\n        if config.stream:\n            return self._arun_stream(start_url, crawler, config)\n        else:\n            return await self._arun_batch(start_url, crawler, config)\n\n    def __call__(self, start_url: str, crawler: AsyncWebCrawler, config: CrawlerRunConfig):\n        return self.arun(start_url, crawler, config)\n\n    @abstractmethod\n    async def shutdown(self) -> None:\n        \"\"\"\n        Clean up resources used by the deep crawl strategy.\n        \"\"\"\n        pass\n\n    @abstractmethod\n    async def can_process_url(self, url: str, depth: int) -> bool:","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/crawl4ai/deep_crawling/base_strategy.py#L82-L118","documentation":"Deep-crawl strategies (BFS/DFS/BestBFS in deep_crawling/base_strategy.py) call crawler.arun(start_url, crawler, config) with a CrawlerRunConfig; passing config=None is rejected up front with this ValueError because every strategy branch (stream or batch) needs link filtering, depths, and stream settings from that config.","triggerScenarios":"Invoking strategy.arun(url, crawler) with two args (config defaults to None); passing config=None explicitly; building a custom loop that calls strategy.__call__ without forwarding the run config; calling DeepCrawlDecorator helpers where the config variable is unset.","commonSituations":"Writing custom deep-crawl strategies that forget to forward config; refactors where CrawlerRunConfig creation moved; calling the strategy directly instead of through crawler.arun(deep_crawl_strategy=...).","solutions":["Always pass a CrawlerRunConfig: await strategy.arun(start_url, crawler, CrawlerRunConfig(...)).","Prefer the high-level path: AsyncWebCrawler with CrawlerRunConfig(deep_cache... deep_crawl_strategy=strategy) so the crawler supplies its own config.","In custom strategies, thread the received config through instead of dropping it."],"exampleFix":"# before\nstrategy = BFSDeepCrawlStrategy(max_depth=2, url_scorer=...\nresults = await strategy.arun(start_url, crawler)  # config=None -> ValueError\n\n# after\nrun_config = CrawlerRunConfig(cache_mode=CacheMode.BYPASS)\nresults = await strategy.arun(start_url, crawler, run_config)","handlingStrategy":"validation","validationCode":"if config is None:\n    config = CrawlerRunConfig(cache_mode=CacheMode.BYPASS)  # supply a default before calling","typeGuard":"from crawl4ai import CrawlerRunConfig\ndef has_run_config(cfg) -> bool:\n    return isinstance(cfg, CrawlerRunConfig)","tryCatchPattern":"try:\n    results = await strategy.arun(start_url, crawler, config)\nexcept ValueError as e:\n    if 'CrawlerRunConfig must be provided' in str(e):\n        results = await strategy.arun(start_url, crawler, CrawlerRunConfig())\n    else:\n        raise","preventionTips":["Always construct CrawlerRunConfig explicitly and thread it through custom strategies.","Prefer crawler.arun(url, config=cfg.with(deep_crawl_strategy=...)) so config is never None.","Add a unit test asserting config forwarding in custom strategy subclasses."],"tags":["deep-crawling","validation","configuration"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}