{"record":{"id":"64f359f4f88e9e81","repo":"unclecode/crawl4ai","slug":"unsupported-import-format-format","errorCode":null,"errorMessage":"Unsupported import format: {format}","messagePattern":"Unsupported import format: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"crawl4ai/adaptive_crawler.py","lineNumber":1878,"sourceCode":"                    if line.strip():\n                        data = json.loads(line)\n                        # Convert back to a mock CrawlResult\n                        mock_result = self._import_dict_to_crawl_result(data)\n                        imported_results.append(mock_result)\n            \n            # Initialize state if needed\n            if not self.state:\n                self.state = CrawlState()\n            \n            # Add imported results\n            self.state.knowledge_base.extend(imported_results)\n            \n            # Update state with imported data\n            await self.strategy.update_state(self.state, imported_results)\n            \n            print(f\"Imported {len(imported_results)} documents from {filepath}\")\n        else:\n            raise ValueError(f\"Unsupported import format: {format}\")\n    \n    def _import_dict_to_crawl_result(self, data: Dict[str, Any]):\n        \"\"\"Convert imported dict back to a mock CrawlResult\"\"\"\n        class MockMarkdown:\n            def __init__(self, content):\n                self.raw_markdown = content\n        \n        class MockCrawlResult:\n            def __init__(self, data):\n                self.url = data.get('url', '')\n                self.markdown = MockMarkdown(data.get('content', ''))\n                self.links = data.get('links', {})\n                self.metadata = data.get('metadata', {})\n                self.success = data.get('success', True)\n                self.timestamp = data.get('timestamp')\n        \n        return MockCrawlResult(data)\n    ","sourceCodeStart":1860,"sourceCodeEnd":1896,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/crawl4ai/adaptive_crawler.py#L1860-L1896","documentation":"EgressBlocked ('URL blocked') from assert_host_allowed when the (lowercased) hostname is in the blocked-hostname set or starts with 'host.docker.internal'. These names resolve to container-host loopback/bridge addresses, which would let a crawled URL reach the host's own services — a classic SSRF vector — so they are rejected by name before DNS resolution.","triggerScenarios":"Crawling http://host.docker.internal:8080/... , localhost/127.0.0.1-style names in the blocked set, or any blocklisted hostname while ALLOW_INTERNAL is off. The name check fires even before getaddrinfo is called.","commonSituations":"Pointing the crawler at a service on the docker host during local testing; reusing docker-compose internal hostnames in the crawl list; a security posture change adding names to _BLOCKED_HOSTNAMES.","solutions":["Crawl the service via its publicly routable address instead of the docker-internal name","For local testing, run a deployment with ALLOW_INTERNAL=true (dev profile) rather than bypassing the broker","Remove container-internal hostnames from production crawl lists"],"exampleFix":"# before\nurls = [\"http://host.docker.internal:9000/health\"]\n\n# after\nurls = [\"https://my-service.example.com/health\"]  # routable, non-internal target","handlingStrategy":"validation","validationCode":"def is_internal_name(host: str) -> bool:\n    h = (host or \"\").lower()\n    return not h or h.startswith(\"host.docker.internal\") or h in BLOCKED_HOSTNAMES","typeGuard":null,"tryCatchPattern":"from egress_broker import EgressBlocked\ntry:\n    assert_host_allowed(host, port)\nexcept EgressBlocked:\n    route_to_internal_crawler(host)  # ALLOW_INTERNAL deployment","preventionTips":["Filter host.docker.internal and known internal names from crawl lists client-side","Expose dev services on public/test hostnames instead of the docker host","Keep an explicitly-scoped ALLOW_INTERNAL deployment for internal targets"],"tags":["ssrf-protection","egress","docker","blocked-hostname"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}