{"record":{"id":"9c25ad0ae4cf38c2","repo":"Panniantong/Agent-Reach","slug":"only-public-http-s-urls-are-allowed","errorCode":null,"errorMessage":"only public HTTP(S) URLs are allowed","messagePattern":"only public HTTP\\(S\\) URLs are allowed","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"agent_reach/utils/url.py","lineNumber":58,"sourceCode":"    try:\n        packed = socket.inet_aton(host)\n    except OSError:\n        return None\n    return ipaddress.IPv4Address(packed)\n\n\ndef normalize_public_http_url(url: str) -> str:\n    \"\"\"Normalize a URL or reject targets that are not clearly public HTTP(S).\"\"\"\n    candidate = str(url or \"\").strip()\n    if (\n        not candidate\n        or \"\\\\\" in candidate\n        or any(\n            character.isspace() or ord(character) < 0x20 or ord(character) == 0x7F\n            for character in candidate\n        )\n    ):\n        raise ValueError(\"only public HTTP(S) URLs are allowed\")\n    if \"://\" not in candidate:\n        candidate = f\"https://{candidate}\"\n\n    try:\n        parsed = urlsplit(candidate)\n        host = (parsed.hostname or \"\").lower().rstrip(\".\")\n        # Accessing the port rejects malformed or out-of-range authorities.\n        _ = parsed.port\n    except (TypeError, ValueError):\n        raise ValueError(\"only public HTTP(S) URLs are allowed\") from None\n\n    literal_address = _literal_ip_address(host)\n    if (\n        parsed.scheme.lower() not in {\"http\", \"https\"}\n        or not host\n        or parsed.username is not None\n        or parsed.password is not None\n        or \"%\" in host","sourceCodeStart":40,"sourceCodeEnd":76,"githubUrl":"https://github.com/Panniantong/Agent-Reach/blob/93ae1d18c37b707dec053c7c4f9d91cd8ef8943d/agent_reach/utils/url.py#L40-L76","documentation":"First rejection branch of normalize_public_http_url: the raw input is not acceptable ASCII-ish text — it is empty, contains a backslash, any whitespace/control character (<0x20 or 0x7F). This is input sanitization before any URL parsing; the strict character whitelist blocks header-injection and parser-confusion tricks.","triggerScenarios":"Passing an empty/None-ish string, a URL with a trailing newline or space ('https://x.com/a\\n'), a URL containing a literal backslash, or embedded tab/CR — common when reading URLs from files, CSVs, clipboard, or LLM output without stripping.","commonSituations":"URLs scraped or generated with surrounding whitespace; copy-paste introducing zero-width or control chars; template strings with accidental newlines; Windows-style backslash paths passed instead of URLs.","solutions":["Strip the input before calling: str(url).strip() and remove surrounding quotes/newlines","Reject or escape backslashes — if the value is a Windows path it is not a URL at all","Sanitize upstream: when URLs come from files/LLMs, filter lines with control characters before passing them in"],"exampleFix":"# before\nurl = open('urls.txt').readline()  # 'https://x.com/a\\n'\nnormalize_public_http_url(url)  # ValueError\n\n# after\nurl = open('urls.txt').readline().strip()\nnormalize_public_http_url(url)","handlingStrategy":"validation","validationCode":"def is_clean_url_input(candidate: str) -> bool:\n    if not candidate:\n        return False\n    if \"\\\\\" in candidate:\n        return False\n    return not any(ord(c) < 0x20 or ord(c) == 0x7F or c.isspace() for c in candidate)","typeGuard":null,"tryCatchPattern":"from agent_reach.utils.url import normalize_public_http_url\ntry:\n    url = normalize_public_http_url(raw)\nexcept ValueError:\n    cleaned = \"\".join(raw.split())  # last-resort whitespace strip\n    url = normalize_public_http_url(cleaned) if cleaned and \"\\\\\" not in cleaned else None","preventionTips":["Always .strip() URLs taken from files, stdin, or LLM output","Reject values containing backslashes early — they are paths, not URLs","Filter lines with control characters when ingesting URL lists"],"tags":["url","validation","input-sanitization"],"backgroundTag":null,"analyzedSha":"93ae1d18c37b707dec053c7c4f9d91cd8ef8943d","analyzedAt":"2026-08-14T22:54:06.735Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}