{"record":{"id":"9d50c4426ad09782","repo":"ytdl-org/youtube-dl","slug":"s-s","errorCode":null,"errorMessage":"%s: %s","messagePattern":"%s: %s","errorType":"exception","errorClass":"ExtractorError","httpStatus":null,"severity":"error","filePath":"youtube_dl/extractor/common.py","lineNumber":700,"sourceCode":"            return self._downloader.urlopen(url_or_request)\n        except tuple(exceptions) as err:\n            if isinstance(err, compat_urllib_error.HTTPError):\n                if self.__can_accept_status_code(err, expected_status):\n                    # Retain reference to error to prevent file object from\n                    # being closed before it can be read. Works around the\n                    # effects of <https://bugs.python.org/issue15002>\n                    # introduced in Python 3.4.1.\n                    err.fp._error = err\n                    return err.fp\n\n            if errnote is False:\n                return False\n            if errnote is None:\n                errnote = 'Unable to download webpage'\n\n            errmsg = '%s: %s' % (errnote, error_to_compat_str(err))\n            if fatal:\n                raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)\n            else:\n                self.report_warning(errmsg)\n                return False\n\n    def _download_webpage_handle(self, url_or_request, video_id, note=None, errnote=None, fatal=True, encoding=None, data=None, headers={}, query={}, expected_status=None):\n        \"\"\"\n        Return a tuple (page content as string, URL handle).\n\n        See _download_webpage docstring for arguments specification.\n        \"\"\"\n        # Strip hashes from the URL (#1038)\n        if isinstance(url_or_request, (compat_str, str)):\n            url_or_request = url_or_request.partition('#')[0]\n\n        urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)\n        if urlh is False:\n            assert not fatal\n            return False","sourceCodeStart":682,"sourceCodeEnd":718,"githubUrl":"https://github.com/ytdl-org/youtube-dl/blob/956b8c585591b401a543e409accb163eeaaa1193/youtube_dl/extractor/common.py#L682-L718","documentation":"This is the generic failure template of _download_webpage internals: when a webpage (or API) download raises a network/HTTP error that is not handled more specifically, youtube-dl builds '<errnote>: <original error>' and, if fatal=True, raises ExtractorError with the underlying exception as cause. 'errnote' defaults to 'Unable to download webpage' but extractors override it (e.g. 'Unable to download API JSON').","triggerScenarios":"Any URLError/HTTPError/socket failure while fetching a URL the extractor requested: DNS failure, connection refused, TLS errors, 403/404/500 responses (unless expected_status was set), timeouts. fatal=False converts it to a warning and returns False instead.","commonSituations":"Dead links (404), bot-blocking WAFs returning 403 to youtube-dl's default User-Agent, corporate proxies/DNS issues, TLS interception problems, transient 5xx.","solutions":["Read the '<original error>' part of the message — it names the real cause (HTTP Error 404, connection reset, etc.).","Set a browser-like User-Agent (--user-agent) and/or pass cookies (--cookies) to pass bot checks.","Check the URL in a browser; 404s mean the link is dead.","For flaky networks, increase --retries/--socket-timeout or use --force-ipv4."],"exampleFix":"# before\nydl.extract(url)  # ERROR: Unable to download webpage: HTTP Error 403: Forbidden\n\n# after\nfrom youtube_dl import YoutubeDL\nwith YoutubeDL({'http_headers': {'User-Agent': 'Mozilla/5.0'}}) as ydl:\n    ydl.extract(url)","handlingStrategy":"try-catch","validationCode":"import urllib.request\ndef url_reachable(url, headers):\n    try:\n        req = urllib.request.Request(url, headers=headers)\n        with urllib.request.urlopen(req, timeout=15) as r:\n            return r.status == 200\n    except Exception:\n        return False","typeGuard":null,"tryCatchPattern":"from youtube_dl.utils import ExtractorError, DownloadError\ntry:\n    ydl.extract_info(url)\nexcept ExtractorError as e:\n    msg = str(e)\n    if 'HTTP Error 403' in msg:\n        retry_with_browser_ua(url)      # bot block\n    elif 'HTTP Error 404' in msg:\n        mark_dead(url)                   # permanent\n    else:\n        raise","preventionTips":["Always parse the '<original error>' suffix to pick the right remedy.","Send a browser User-Agent and cookies for sites with bot protection.","Pre-validate URLs (status 200) in batch pipelines to fail fast."],"tags":["network","http-error","download-webpage","user-agent"],"backgroundTag":null,"analyzedSha":"956b8c585591b401a543e409accb163eeaaa1193","analyzedAt":"2026-08-14T18:59:47.863Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}