ComposioHQ/composio · error · BlockedInternalUrlError

Refusing to fetch a malformed or non-http(s) URL

Error message

Refusing to fetch a malformed or non-http(s) URL

What it means

assert_safe_fetch_target prepares the URL with requests and requires an http/https scheme plus a hostname; if URL preparation/parsing fails or those checks fail, BlockedInternalUrlError is raised wrapping the underlying error. This is SSRF protection — the SDK refuses to fetch malformed or non-HTTP URLs (ftp://, file://, bare hostnames without parseable hostname, etc.).

Source

Thrown at python/composio/utils/url_safety.py:92

def assert_safe_fetch_target(url: str) -> t.List[str]:
    """Refuse non-HTTP(S) URLs and hosts that resolve to internal addresses.

    Parse the URL after Requests prepares it so validation uses the same
    canonical hostname that the eventual connection will use.

    :returns: The validated addresses to connect to, in resolver order.
        Callers must connect to *these* rather than re-resolving the hostname;
        see :func:`safe_get`.
    """
    try:
        prepared_url = requests.Request(method="GET", url=url).prepare().url
        if prepared_url is None:
            raise ValueError("Prepared URL is missing")
        parsed = urlparse(prepared_url)
        if parsed.scheme not in {"http", "https"} or not parsed.hostname:
            raise ValueError("URL must use HTTP(S) and include a hostname")
    except (requests.exceptions.RequestException, ValueError):
        raise BlockedInternalUrlError(
            "Refusing to fetch a malformed or non-http(s) URL"
        ) from None

    try:
        # Resolver order is kept: it encodes the system's address preference
        # (RFC 6724), and connecting walks it the way urllib3 would.
        addresses: t.List[str] = []
        for result in socket.getaddrinfo(parsed.hostname, None):
            address = result[4][0]
            if not isinstance(address, str):
                raise BlockedInternalUrlError(
                    f'Could not resolve host "{parsed.hostname}"'
                )
            if address not in addresses:
                addresses.append(address)
    except socket.gaierror as error:
        raise BlockedInternalUrlError(
            f'Could not resolve host "{parsed.hostname}"'

View on GitHub (pinned to 64b1b85502)

Solutions

1. Normalize URLs before fetching: strip whitespace/quotes, prepend https:// when the scheme is missing, and reject non-http(s) schemes early
2. Validate with urlparse yourself: scheme in {http, https} and a non-empty hostname
3. If the target is genuinely http(s) but still blocked, print the exact string being passed — hidden characters (zero-width, \r) are a common cause
4. Whitelist expected hosts in your own layer and construct URLs from trusted templates rather than raw input

Example fix

# before
fetch("example.com/data.json")
# after
from urllib.parse import urlparse
url = "example.com/data.json"
if not urlparse(url).scheme:
    url = "https://" + url
assert urlparse(url).scheme in {"http", "https"} and urlparse(url).hostname
fetch(url)

from urllib.parse import urlparse
def normalize_and_check(url):
    url = url.strip().strip('"\'')
    if not urlparse(url).scheme:
        url = "https://" + url
    p = urlparse(url)
    if p.scheme not in {"http", "https"} or not p.hostname:
        raise ValueError(f"unsafe/malformed URL: {url!r}")
    return url

def is_fetchable_url(u: str) -> bool:
    try:
        p = urlparse(u.strip())
        return p.scheme in {"http", "https"} and bool(p.hostname)
    except ValueError:
        return False

from composio.utils.url_safety import BlockedInternalUrlError
try:
    fetch(url)
except BlockedInternalUrlError:
    url = normalize_and_check(url)
    fetch(url)

When it happens

Trigger: Passing a URL like "ftp://example.com/file", "file:///etc/passwd", "example.com/path" (no scheme, so hostname parsing fails), or a malformed URL that requests' preparation rejects (e.g. invalid characters, bad percent-encoding). The check runs before any DNS resolution in _pinned_request.

Common situations: User- or LLM-supplied URLs not normalized (missing https:// prefix); configs containing internal schemes; copy-pasted URLs with stray whitespace/quotes that break parsing; data sources mixing URNs/paths with URLs.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Refusing to fetch a malformed URL
• Refusing to fetch a non-http(s) URL (scheme "${url.protocol}
• Refusing to fetch "${host}" — it resolves to a private, loop
• Refusing to fetch "{parsed.hostname}" because it resolves to
• Refusing to talk to "{hostname}": the connection was establi

AI-assisted analysis of ComposioHQ/composio@64b1b85502 (2026-08-28). Data as JSON: /api/errors/3b9c9874c82bed20. Report an issue: GitHub.