{"record":{"id":"dff4eb125d54434d","repo":"unclecode/crawl4ai","slug":"file-not-found-filepath","errorCode":null,"errorMessage":"File not found: {filepath}","messagePattern":"File not found: (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"warning","filePath":"crawl4ai/adaptive_crawler.py","lineNumber":1854,"sourceCode":"        if self.state:\n            export_dict['crawl_metadata'] = {\n                'crawl_order': self.state.crawl_order.index(export_dict['url']) + 1 if export_dict['url'] in self.state.crawl_order else 0,\n                'confidence_at_crawl': self.state.metrics.get('confidence', 0),\n                'total_documents': self.state.total_documents\n            }\n        \n        return export_dict\n    \n    async def import_knowledge_base(self, filepath: Union[str, Path], format: str = \"jsonl\") -> None:\n        \"\"\"Import a knowledge base from a file\n        \n        Args:\n            filepath: Path to the file to import\n            format: Import format - currently supports 'jsonl'\n        \"\"\"\n        filepath = Path(filepath)\n        if not filepath.exists():\n            raise FileNotFoundError(f\"File not found: {filepath}\")\n        \n        if format == \"jsonl\":\n            imported_results = []\n            with open(filepath, 'r', encoding='utf-8') as f:\n                for line in f:\n                    if line.strip():\n                        data = json.loads(line)\n                        # Convert back to a mock CrawlResult\n                        mock_result = self._import_dict_to_crawl_result(data)\n                        imported_results.append(mock_result)\n            \n            # Initialize state if needed\n            if not self.state:\n                self.state = CrawlState()\n            \n            # Add imported results\n            self.state.knowledge_base.extend(imported_results)\n            ","sourceCodeStart":1836,"sourceCodeEnd":1872,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/crawl4ai/adaptive_crawler.py#L1836-L1872","documentation":"EgressBlocked ('URL blocked') from assert_host_allowed when the host argument is empty/None. The broker treats a missing hostname as unconditionally blocked — there is nothing to resolve and the request must not proceed. It is a guard against callers passing malformed URLs downstream of parsing.","triggerScenarios":"Calling assert_host_allowed('', port) or with None — typically because a URL string like 'http://', ':///path', or a parse artifact yielded an empty host. Also hit when code extracts host from user input without validating it first.","commonSituations":"User-submitted URL lists containing malformed entries; URL construction bugs producing scheme-only strings; empty-string defaults flowing in from config.","solutions":["Validate/normalize URLs before calling the egress layer: require a scheme and a non-empty hostname","Filter empty or scheme-only entries out of batch crawl lists client-side","Log the offending raw input where you validate so the malformed entry is identifiable"],"exampleFix":"# before\nassert_host_allowed(urlparse(raw).hostname or \"\", 443)\n\n# after\nfrom urllib.parse import urlparse\np = urlparse(raw)\nif not p.hostname:\n    raise ValueError(f\"malformed URL, no host: {raw!r}\")\nassert_host_allowed(p.hostname, p.port or 443)","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"from urllib.parse import urlparse\ndef has_http_host(url) -> bool:\n    try:\n        return bool(urlparse(str(url)).hostname)\n    except ValueError:\n        return False","tryCatchPattern":"if not has_http_host(url):\n    raise ValueError(f\"URL missing host: {url!r}\")\nassert_host_allowed(urlparse(url).hostname, urlparse(url).port or 80)","preventionTips":["Validate URLs (scheme + host present) at intake, before the egress layer","Never pass parsed fragments or relative URLs to crawl APIs","Log the raw input on validation failure to find the malformed source"],"tags":["ssrf-protection","egress","url-validation","url-blocked"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-16T03:17:38.424Z"}