{"record":{"id":"5061d407ba3ea87f","repo":"unslothai/unsloth","slug":"youtube-returned-a-caption-url-from-an-unexpected","errorCode":null,"errorMessage":"YouTube returned a caption URL from an unexpected host.","messagePattern":"YouTube returned a caption URL from an unexpected host\\.","errorType":"exception","errorClass":"TranscriptUnavailable","httpStatus":null,"severity":"error","filePath":"studio/backend/core/youtube_transcript.py","lineNumber":234,"sourceCode":"def _default_track_index(tracks: list[dict[str, Any]], tracklist: dict[str, Any]) -> int:\n    \"\"\"Index of the caption track paired with the video's default audio track.\n\n    A multi-language video lists its tracks alphabetically, so track 0 is often an\n    unrelated translation rather than the language actually spoken.\n    \"\"\"\n    audio_tracks = tracklist.get(\"audioTracks\") or []\n    audio_index = tracklist.get(\"defaultAudioTrackIndex\")\n    if isinstance(audio_index, int) and 0 <= audio_index < len(audio_tracks):\n        caption_index = (audio_tracks[audio_index] or {}).get(\"defaultCaptionTrackIndex\")\n        if isinstance(caption_index, int) and 0 <= caption_index < len(tracks):\n            return caption_index\n    return 0\n\n\ndef _validated_caption_url(url: str) -> SplitResult:\n    parsed = urlsplit(url)\n    if parsed.scheme != \"https\" or (parsed.hostname or \"\").lower() not in _CAPTION_HOSTS:\n        raise TranscriptUnavailable(\"YouTube returned a caption URL from an unexpected host.\")\n    return parsed\n\n\ndef _caption_url(base_url: str) -> str:\n    \"\"\"Ask a caption baseUrl for json3, keeping the blank-valued params YouTube sends.\"\"\"\n    parsed = _validated_caption_url(base_url)\n    query = parse_qs(parsed.query, keep_blank_values = True)\n    query[\"fmt\"] = [\"json3\"]\n    return urlunsplit(parsed._replace(query = urlencode(query, doseq = True)))\n\n\nasync def _read_capped(response: httpx.Response, limit: int, message: str) -> bytes:\n    body = bytearray()\n    async for chunk in response.aiter_bytes():\n        body.extend(chunk)\n        if len(body) > limit:\n            raise TranscriptUnavailable(message)\n    return bytes(body)","sourceCodeStart":216,"sourceCodeEnd":252,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/youtube_transcript.py#L216-L252","documentation":"TranscriptUnavailable raised by _validated_caption_url when a caption baseUrl (or a redirect target) is not an https URL whose hostname is in the allowlist _CAPTION_HOSTS. This is a defense so caption fetching only ever talks to trusted YouTube caption hosts, never to arbitrary hosts injected into player data.","triggerScenarios":"A caption track's baseUrl uses http instead of https, or points at a non-allowlisted host (e.g. a CDN change like a new googlevideo subdomain not in _CAPTION_HOSTS, or tampered/proxied player data); a redirect Location header leading off-host.","commonSituations":"YouTube rotating caption CDN hostnames after an update (allowlist goes stale); environments with DNS/proxy manipulation rewriting URLs.","solutions":["Update the _CAPTION_HOSTS allowlist in core/youtube_transcript.py with the new legitimate YouTube caption host (verify the host independently first).","Verify nothing on the network path (proxy, TLS-terminating middlebox) rewrites the baseUrl.","If the URL came from tampered data, discard that player response and re-fetch."],"exampleFix":"# before\n_CAPTION_HOSTS = {\"www.youtube.com\"}  # new host youtube.googleapis.com rejected\n\n# after\n_CAPTION_HOSTS = {\"www.youtube.com\", \"youtube.googleapis.com\", \"www.googlevideo.com\"}","handlingStrategy":"try-catch","validationCode":"from urllib.parse import urlsplit\n\nCAPTION_HOSTS = {\"www.youtube.com\"}  # keep in sync with core/youtube_transcript.py\n\ndef is_trusted_caption_url(url: str) -> bool:\n    p = urlsplit(url)\n    return p.scheme == \"https\" and (p.hostname or \"\").lower() in CAPTION_HOSTS","typeGuard":"def is_valid_caption_url(url: str) -> bool:\n    p = urlsplit(url)\n    return p.scheme == \"https\" and (p.hostname or \"\").lower() in _CAPTION_HOSTS","tryCatchPattern":"try:\n    transcript = await fetch_transcript(video_id)\nexcept TranscriptUnavailable as e:\n    if \"unexpected host\" in str(e):\n        logger.error(\"caption host allowlist may be stale: %s\", e)\n    raise","preventionTips":["Keep the _CAPTION_HOSTS allowlist updated when YouTube rotates caption CDNs.","Never disable the host check to 'make it work' — it exists to prevent SSRF via tampered player data.","If a proxy rewrites URLs on your network, exempt YouTube caption traffic."],"tags":["youtube","transcript","security","url-validation"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}