{"record":{"id":"30e19621ab9e72d7","repo":"assafelovic/gpt-researcher","slug":"url-host-host-r-resolves-to-a-non-public-address","errorCode":null,"errorMessage":"URL host {host!r} resolves to a non-public address ({ip_str}); set ALLOW_PRIVATE_URLS=true to allow internal targets.","messagePattern":"URL host (.+?) resolves to a non-public address \\((.+?)\\); set ALLOW_PRIVATE_URLS=true to allow internal targets\\.","errorType":"validation","errorClass":"UnsafeURLError","httpStatus":null,"severity":"error","filePath":"gpt_researcher/utils/url_security.py","lineNumber":111,"sourceCode":"        allow_private = _private_urls_allowed()\n    if allow_private:\n        return url\n\n    try:\n        addrinfo = socket.getaddrinfo(host, None)\n    except socket.gaierror as exc:\n        raise UnsafeURLError(f\"Could not resolve host {host!r}: {exc}\") from exc\n\n    for info in addrinfo:\n        ip_str = info[4][0]\n        try:\n            ip = ipaddress.ip_address(ip_str)\n        except ValueError as exc:\n            raise UnsafeURLError(\n                f\"Host {host!r} resolved to an invalid address {ip_str!r}.\"\n            ) from exc\n        if _is_disallowed_ip(ip):\n            raise UnsafeURLError(\n                f\"URL host {host!r} resolves to a non-public address ({ip_str}); \"\n                \"set ALLOW_PRIVATE_URLS=true to allow internal targets.\"\n            )\n\n    return url\n\n\ndef is_safe_url(url: str, *, allow_private: bool | None = None) -> bool:\n    \"\"\"Return ``True`` if ``url`` passes :func:`validate_url`, else ``False``.\"\"\"\n    try:\n        validate_url(url, allow_private=allow_private)\n        return True\n    except UnsafeURLError:\n        return False\n","sourceCodeStart":93,"sourceCodeEnd":126,"githubUrl":"https://github.com/assafelovic/gpt-researcher/blob/6f998577d547b1e54ec662dac63583aa11e3b84b/gpt_researcher/utils/url_security.py#L93-L126","documentation":"validate_url resolves the hostname and rejects any address that is private, loopback, link-local, or otherwise non-public, to prevent SSRF against internal services. Setting ALLOW_PRIVATE_URLS=true (or passing allow_private=True) bypasses the check.","triggerScenarios":"Fetching http://localhost:8000, http://127.0.0.1, http://192.168.x.x, http://10.x.x.x, 169.254.169.254 (cloud metadata), or a public name that DNS-resolves to a private IP, without ALLOW_PRIVATE_URLS set.","commonSituations":"Developers testing against a local dev server or internal service; deployments behind NAT where a public hostname resolves internally; intentional intranet crawling in Docker/Kubernetes.","solutions":["Set env var ALLOW_PRIVATE_URLS=true before creating the researcher (only in trusted environments — it disables SSRF protection).","Expose the local service on a public hostname/IP and fetch that instead.","Pass allow_private=True programmatically to validate_url if calling it directly."],"exampleFix":"# before\nresearcher = GPTResearcher(query=\"...\", report_source=\"web\")  # fetching localhost fails\n# after\nimport os\nos.environ[\"ALLOW_PRIVATE_URLS\"] = \"true\"\nresearcher = GPTResearcher(query=\"...\", report_source=\"web\")","handlingStrategy":"fallback","validationCode":"import ipaddress, socket\nfrom urllib.parse import urlparse\n\ndef is_public(url: str) -> bool:\n    host = urlparse(url).hostname\n    if not host:\n        return False\n    try:\n        return all(not ipaddress.ip_address(i[4][0]).is_private for i in socket.getaddrinfo(host, None))\n    except socket.gaierror:\n        return False","typeGuard":null,"tryCatchPattern":"from gpt_researcher.utils.url_security import UnsafeURLError\ntry:\n    validate_url(url)\nexcept UnsafeURLError as e:\n    if \"non-public address\" in str(e) and TRUST_INTERNAL:\n        os.environ[\"ALLOW_PRIVATE_URLS\"] = \"true\"  # then retry deliberately\n    else:\n        raise","preventionTips":["Set ALLOW_PRIVATE_URLS=true only in trusted, isolated environments.","Prefer exposing internal services via public hostnames rather than disabling the guard.","Never let end-user input pick URLs without this validation."],"tags":["ssrf-protection","network","security","python"],"backgroundTag":"private-ip-blocked-by-ssrf-guard","analyzedSha":"6f998577d547b1e54ec662dac63583aa11e3b84b","analyzedAt":"2026-08-28T17:50:07.383Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}