{"record":{"id":"98616db5bf353594","repo":"Panniantong/Agent-Reach","slug":"ssrf-blocked-url-host-is-invalid","errorCode":null,"errorMessage":"SSRF blocked: URL host is invalid","messagePattern":"SSRF blocked: URL host is invalid","errorType":"exception","errorClass":"TranscribeError","httpStatus":null,"severity":"error","filePath":"agent_reach/transcribe.py","lineNumber":243,"sourceCode":"        normalized_url = url\n        parsed = urlparse(url)\n        if parsed.scheme not in {\"http\", \"https\"}:\n            raise TranscribeError(\"SSRF blocked: only public http(s) URLs are allowed\")\n\n    raw_authority = normalized_url.split(\"://\", 1)[1]\n    raw_authority = raw_authority.split(\"/\", 1)[0]\n    raw_authority = raw_authority.split(\"?\", 1)[0]\n    raw_authority = raw_authority.split(\"#\", 1)[0]\n    if \"\\\\\" in raw_authority or \"%\" in raw_authority:\n        raise TranscribeError(\"SSRF blocked: encoded or ambiguous URL host\")\n\n    raw_host = (parsed.hostname or \"\").strip().rstrip(\".\")\n    if not raw_host:\n        raise TranscribeError(\"SSRF blocked: URL host is missing\")\n    try:\n        host = raw_host.encode(\"idna\").decode(\"ascii\").lower().rstrip(\".\")\n    except UnicodeError:\n        raise TranscribeError(\"SSRF blocked: URL host is invalid\") from None\n    if host in _BLOCKED_HOSTS or host.endswith(\".localhost\"):\n        raise TranscribeError(\"SSRF blocked: internal host is not allowed\")\n    if _is_private_ip(host):\n        raise TranscribeError(\"SSRF blocked: private/internal IP is not allowed\")\n\n\ndef download_audio(url: str, out_dir: Path) -> Path:\n    \"\"\"Download audio with yt-dlp into out_dir; return the resulting file path.\"\"\"\n    _assert_safe_public_url(url)\n    _require(\"yt-dlp\")\n    template = out_dir / \"source.%(ext)s\"\n    _run(\n        [\n            \"yt-dlp\",\n            \"-x\",\n            \"--audio-format\",\n            \"m4a\",\n            \"--audio-quality\",","sourceCodeStart":225,"sourceCodeEnd":261,"githubUrl":"https://github.com/Panniantong/Agent-Reach/blob/93ae1d18c37b707dec053c7c4f9d91cd8ef8943d/agent_reach/transcribe.py#L225-L261","documentation":"Raised by _assert_safe_public_url (transcribe.py:240-243) when the extracted hostname fails IDNA encoding (raw_host.encode('idna') raises UnicodeError). Hosts with disallowed Unicode codepoints, empty labels, or overlong labels cannot be safely canonicalized, so the guard treats them as invalid rather than passing ambiguous bytes to yt-dlp.","triggerScenarios":"Hostnames containing characters IDNA rejects: 'https://exa mple.com/' (space), 'https://exa​mple.com/' (zero-width char), labels longer than 63 chars, consecutive dots ('example..com'), or non-NFC Unicode that str.encode('idna') refuses.","commonSituations":"Copy-paste of internationalized domains with stray invisible characters; LLM-generated URLs with typographic dashes (en/em dash) instead of ASCII hyphens; corrupted strings from upstream data sources.","solutions":["Clean the host: strip zero-width/whitespace characters, normalize to NFC, replace typographic dashes with '-'","Use the punycode form of IDN hosts (e.g. 'xn--bcher-kva.example' instead of 'bücher.example' if encoding keeps failing)","Validate hosts against a strict regex (^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)+$) before calling transcribe()"],"exampleFix":"# before\nurl = \"https://exa​mple.com/audio.mp3\"  # zero-width space -> SSRF blocked: URL host is invalid\n\n# after\nimport unicodedata\nhost = unicodedata.normalize(\"NFKC\", host).replace(\"\\u200b\", \"\").replace(\"\\u2013\", \"-\")\nurl = f\"https://{host}/audio.mp3\"","handlingStrategy":"validation","validationCode":"import unicodedata\n\ndef host_encodes_idna(host: str) -> bool:\n    host = host.strip().rstrip(\".\")\n    try:\n        host.encode(\"idna\")\n        return True\n    except UnicodeError:\n        return False\n\ndef clean_host(host: str) -> str:\n    return (unicodedata.normalize(\"NFKC\", host)\n            .replace(\"\\u200b\", \"\").replace(\"\\u2013\", \"-\").replace(\" \", \"\"))","typeGuard":null,"tryCatchPattern":"from agent_reach.transcribe import TranscribeError\ntry:\n    transcribe(url)\nexcept TranscribeError as e:\n    if \"URL host is invalid\" in str(e):\n        return transcribe(url_with_cleaned_host(url))  # NFKC-normalize, strip invisibles\n    raise","preventionTips":["Normalize hostnames to NFKC and strip zero-width characters before use","Use punycode for internationalized domains","Validate hosts against a strict hostname regex at input boundaries"],"tags":["ssrf","url-validation","idna","unicode","security"],"backgroundTag":null,"analyzedSha":"93ae1d18c37b707dec053c7c4f9d91cd8ef8943d","analyzedAt":"2026-08-14T22:54:06.735Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}