assafelovic/gpt-researcher · error · UnsafeURLError

Could not resolve host {host!r}: {exc}

Error message

Could not resolve host {host!r}: {exc}

What it means

Raised when validate_url cannot DNS-resolve the URL's hostname (socket.getaddrinfo raises gaierror). Because gpt-researcher must resolve the host to verify it points at a public IP (SSRF mitigation), an unresolvable host aborts the fetch.

Source

Thrown at gpt_researcher/utils/url_security.py:100

    if scheme not in ALLOWED_SCHEMES:
        raise UnsafeURLError(
            f"URL scheme {scheme or '(none)'!r} is not allowed; "
            "only http and https URLs may be fetched."
        )

    host = parsed.hostname
    if not host:
        raise UnsafeURLError("URL must include a valid host.")

    if allow_private is None:
        allow_private = _private_urls_allowed()
    if allow_private:
        return url

    try:
        addrinfo = socket.getaddrinfo(host, None)
    except socket.gaierror as exc:
        raise UnsafeURLError(f"Could not resolve host {host!r}: {exc}") from exc

    for info in addrinfo:
        ip_str = info[4][0]
        try:
            ip = ipaddress.ip_address(ip_str)
        except ValueError as exc:
            raise UnsafeURLError(
                f"Host {host!r} resolved to an invalid address {ip_str!r}."
            ) from exc
        if _is_disallowed_ip(ip):
            raise UnsafeURLError(
                f"URL host {host!r} resolves to a non-public address ({ip_str}); "
                "set ALLOW_PRIVATE_URLS=true to allow internal targets."
            )

    return url

View on GitHub (pinned to 6f998577d5)

Solutions

1. Verify the hostname resolves: run 'nslookup <host>' or 'socket.getaddrinfo(host, None)' in the same environment.
2. If the target is an internal host you trust, set ALLOW_PRIVATE_URLS=true to skip resolution/public-IP checks.
3. Fix typos in the URL or use an IP/public DNS-resolvable name.
4. In tests, mock socket.getaddrinfo or gpt_researcher.utils.url_security.validate_url.

Example fix

# before
await researcher.extract_data_from_url("https://internal-svc.local/doc")
# after (trusted internal target)
os.environ["ALLOW_PRIVATE_URLS"] = "true"
await researcher.extract_data_from_url("https://internal-svc.local/doc")

import socket

def resolvable(host: str) -> bool:
    try:
        socket.getaddrinfo(host, None)
        return True
    except socket.gaierror:
        return False

from gpt_researcher.utils.url_security import UnsafeURLError
try:
    validate_url(url)
except UnsafeURLError as e:
    if "Could not resolve" in str(e):
        logger.warning("DNS failure for %s; skipping", url)
    else:
        raise

When it happens

Trigger: Calling extract_data_from_url / _download_and_process / validate_url with a hostname that fails DNS resolution: typo'd domain, internal-only DNS name while ALLOW_PRIVATE_URLS is unset, or a sandboxed environment with no network/DNS access.

Common situations: CI containers with restricted DNS, air-gapped test environments (tests mock this with patching), typo'd domains, or '.local'/.internal hostnames in cluster deployments.

Understand the failure class

• DNS resolution errors: ENOTFOUND and getaddrinfo failures — how hostname lookups fail and how to debug them.

Related errors

• Host {host!r} resolved to an invalid address {ip_str!r}.
• URL host {host!r} resolves to a non-public address ({ip_str}
• URL must include a valid host.
• Cost must be an integer or float
• Embedding provider not found.

AI-assisted analysis of assafelovic/gpt-researcher@6f998577d5 (2026-08-28). Data as JSON: /api/errors/c4c3c84197185ff7. Report an issue: GitHub.