{"record":{"id":"a7e169ff3c499d89","repo":"dotnet/yarp","slug":"failed-to-download-url-after-max-retries-attem","errorCode":null,"errorMessage":"Failed to download {url} after {max_retries} attempts.","messagePattern":"Failed to download (.+?) after (.+?) attempts\\.","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"eng/common/cross/install-debs.py","lineNumber":47,"sourceCode":"\n                        # verify checksum if provided\n                        if checksum:\n                            sha256 = hashlib.sha256(content).hexdigest()\n                            if sha256 != checksum:\n                                raise Exception(f\"SHA256 mismatch for {url}: expected {checksum}, got {sha256}\")\n\n                        f.write(content)\n                    print(f\"Downloaded {url} at {dest_path}\")\n                    return\n                else:\n                    raise Exception(f\"Failed to download {url}, Status Code: {response.status}\")\n        except (asyncio.CancelledError, asyncio.TimeoutError, aiohttp.ClientError) as e:\n            print(f\"Error downloading {url}: {type(e).__name__} - {e}. Retrying...\")\n\n        attempt += 1\n        await asyncio.sleep(retry_delay)\n\n    raise Exception(f\"Failed to download {url} after {max_retries} attempts.\")\n\nasync def download_deb_files_parallel(mirror, packages, tmp_dir):\n    \"\"\"Download .deb files in parallel.\"\"\"\n    os.makedirs(tmp_dir, exist_ok=True)\n\n    tasks = []\n    timeout = aiohttp.ClientTimeout(total=60)\n    async with aiohttp.ClientSession(timeout=timeout) as session:\n        for pkg, info in packages.items():\n            filename = info.get(\"Filename\")\n            if filename:\n                url = f\"{mirror}/{filename}\"\n                dest_path = os.path.join(tmp_dir, os.path.basename(filename))\n                tasks.append(asyncio.create_task(download_file(session, url, dest_path, checksum=info.get(\"SHA256\"))))\n\n        await asyncio.gather(*tasks)\n\nasync def download_package_index_parallel(mirror, arch, suites, check_sig, keyring):","sourceCodeStart":29,"sourceCodeEnd":65,"githubUrl":"https://github.com/dotnet/yarp/blob/bd11867bee7df522e7fd3effb08a9c85fd616908/eng/common/cross/install-debs.py#L29-L65","documentation":"This exception is raised at the bottom of download_file after the retry loop exhausts all max_retries (default 3) attempts. It is only reached when every attempt failed with an exception type listed in the except clause: asyncio.CancelledError, asyncio.TimeoutError, or aiohttp.ClientError (connection refused, DNS failure, SSL error, etc.). Note that HTTP non-200 responses raise a different exception (error 103) that bypasses this path entirely because it is not caught by the except clause.","triggerScenarios":"download_file's while loop (lines 23-45) iterates max_retries times (default 3). On each attempt, session.get raises asyncio.TimeoutError (the 60-second timeout fires), aiohttp.ClientConnectorError (DNS resolution failure, connection refused, TLS handshake failure), or asyncio.CancelledError. After all attempts, control falls through to line 47 and raises this final exception.","commonSituations":"DNS resolution failure for the mirror hostname; the mirror host is unreachable (firewall, network partition, VPN routing issue); TLS/SSL certificate problems causing handshake failures; the default 60-second timeout is too short for a slow mirror or large packages; all connection attempts time out behind a restrictive corporate proxy; the mirror is down or rate-limiting all connections; retry_delay (2 seconds) is too short for the mirror to recover between attempts.","solutions":["Verify network connectivity to the mirror: run curl -v <mirror-url> or ping the mirror host to confirm reachability.","Increase max_retries and retry_delay in the download_file call (e.g. max_retries=5, retry_delay=5) if the mirror is intermittently available.","Check DNS resolution: ensure the mirror hostname resolves correctly (dig <hostname> or nslookup).","If behind a corporate proxy, set the HTTPS_PROXY / HTTP_PROXY environment variables so aiohttp routes through it.","Increase the per-request timeout (currently 60s) if downloading large .deb files from a slow mirror.","Switch to a faster, closer, or more reliable mirror."],"exampleFix":"# before -- default retry parameters may be too conservative\ntasks.append(asyncio.create_task(download_file(session, url, dest_path, checksum=info.get(\"SHA256\"))))\n\n# after -- pass more generous retry parameters for flaky mirrors\ntasks.append(asyncio.create_task(\n    download_file(session, url, dest_path,\n                  max_retries=5, retry_delay=5, timeout=120,\n                  checksum=info.get(\"SHA256\"))))","handlingStrategy":"retry","validationCode":"# Pre-check network connectivity and DNS before starting downloads.\nimport socket, urllib.parse\nparsed = urllib.parse.urlparse(mirror)\ntry:\n    socket.gethostbyname(parsed.hostname)\n    print(f\"DNS resolves for {parsed.hostname}\")\nexcept socket.gaierror:\n    print(f\"ERROR: Cannot resolve {parsed.hostname}. Check DNS or mirror URL.\")","typeGuard":null,"tryCatchPattern":"# Catch the final failure and provide a clear diagnostic with retry guidance\ntry:\n    await asyncio.gather(*tasks)\nexcept Exception as e:\n    if 'after' in str(e) and 'attempts' in str(e):\n        print(f\"All retries exhausted. Possible causes: network down, DNS failure, \"\n              f\"TLS error, or mirror unreachable.\")\n        print(f\"Try: increasing max_retries, checking proxy settings, or switching mirrors.\")\n    raise","preventionTips":["Set generous retry parameters (max_retries=5, retry_delay=5, timeout=120) for unreliable networks.","Verify DNS resolution and network reachability of the mirror before starting the build.","If behind a corporate proxy, set HTTP_PROXY/HTTPS_PROXY environment variables.","Use a mirror geographically close to reduce latency and timeout risk.","Monitor download progress to detect stalls early."],"tags":["python","debian","download","retry","timeout","network","apt"],"backgroundTag":null,"analyzedSha":"bd11867bee7df522e7fd3effb08a9c85fd616908","analyzedAt":"2026-08-13T21:29:49.359Z","schemaVersion":2},"datasetVersion":"2026-08-14T00:17:13.853Z"}