{"record":{"id":"d1f09a0dd446e4e6","repo":"unclecode/crawl4ai","slug":"invalid-url-make-sure-the-url-is-a-non-empty-stri-d1f09a","errorCode":null,"errorMessage":"Invalid URL, make sure the URL is a non-empty string","messagePattern":"Invalid URL, make sure the URL is a non-empty string","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"crawl4ai/async_webcrawler.py:252","lineNumber":1595,"sourceCode":"            )\n            result = await crawler.arun(url=\"https://example.com\", crawler_config=config)\n\n        Args:\n            url: The URL to crawl (http://, https://, file://, or raw:)\n            crawler_config: Configuration object controlling crawl behavior\n            [other parameters maintained for backwards compatibility]\n\n        Returns:\n            CrawlResult: The result of crawling and processing\n        \"\"\"\n        # Auto-start if not ready\n        if not self.ready:\n            await self.start()\n\n        config = config or CrawlerRunConfig()\n        if not isinstance(url, str) or not url:\n            raise ValueError(\n                \"Invalid URL, make sure the URL is a non-empty string\")\n\n        async with self._lock or self.nullcontext():\n            try:\n                self.logger.verbose = config.verbose\n\n                # Default to ENABLED if no cache mode specified\n                if config.cache_mode is None:\n                    config.cache_mode = CacheMode.ENABLED\n\n                # Create cache context\n                cache_context = CacheContext(url, config.cache_mode, False)\n\n                # Initialize processing variables\n                async_response: AsyncCrawlResponse = None\n                cached_result: CrawlResult = None\n                screenshot_data = None\n                pdf_data = None\n                extracted_content = None","sourceCodeStart":1577,"sourceCodeEnd":1613,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/deploy/docker/c4ai-code-context.md#L1577-L1613","documentation":"ValueError from AsyncWebCrawler's crawl entry (apriori path): url must be a non-empty str. Raised after auto-start, before any lock acquisition or cache handling, so it fails fast on bad input.","triggerScenarios":"await crawler.arun(url='') or url=None (or an int/list) - typically a URL built from a variable that ended up empty, or a loop iterating a list with an empty element.","commonSituations":"URLs scraped from a page where an anchor href was empty; CSV/database rows with blank URL columns; a chain that filters URLs and accidentally yields None; a positional argument shifted so a non-URL value lands in url.","solutions":["Filter inputs first: urls = [u for u in urls if isinstance(u, str) and u.strip()]","If a blank URL is expected data, log and skip it rather than aborting the whole batch","Double-check the arun/apriori call signature - a misplaced argument can land in url="],"exampleFix":"# before\nfor u in rows:\n    result = await crawler.arun(url=u, config=config)\n\n# after\nfor u in rows:\n    if not isinstance(u, str) or not u.strip():\n        logger.warning(\"skipping empty url row\")\n        continue\n    result = await crawler.arun(url=u, config=config)","handlingStrategy":"validation","validationCode":"assert isinstance(url, str) and url.strip(), f\"url must be a non-empty string, got {url!r}\"","typeGuard":"def is_crawlable_url_value(u) -> bool:\n    return isinstance(u, str) and bool(u.strip())","tryCatchPattern":"try:\n    result = await crawler.arun(url=url, config=config)\nexcept ValueError as e:\n    if \"non-empty string\" in str(e):\n        logger.warning(\"skipping bad url %r\", url)\n        result = None","preventionTips":["Sanitize URL columns at ingest time (drop or repair empties)","In batch loops, filter with isinstance+strip checks before calling arun"],"tags":["crawl4ai","validation","url","crawler"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}