{"record":{"id":"6b10c30ff60cbd35","repo":"dotnet/aspnetcore","slug":"failed-to-download-url-after-max-retries-attem","errorCode":null,"errorMessage":"Failed to download {url} after {max_retries} attempts.","messagePattern":"Failed to download (.+?) after (.+?) attempts\\.","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"eng/common/cross/install-debs.py","lineNumber":47,"sourceCode":"\n                        # verify checksum if provided\n                        if checksum:\n                            sha256 = hashlib.sha256(content).hexdigest()\n                            if sha256 != checksum:\n                                raise Exception(f\"SHA256 mismatch for {url}: expected {checksum}, got {sha256}\")\n\n                        f.write(content)\n                    print(f\"Downloaded {url} at {dest_path}\")\n                    return\n                else:\n                    raise Exception(f\"Failed to download {url}, Status Code: {response.status}\")\n        except (asyncio.CancelledError, asyncio.TimeoutError, aiohttp.ClientError) as e:\n            print(f\"Error downloading {url}: {type(e).__name__} - {e}. Retrying...\")\n\n        attempt += 1\n        await asyncio.sleep(retry_delay)\n\n    raise Exception(f\"Failed to download {url} after {max_retries} attempts.\")\n\nasync def download_deb_files_parallel(mirror, packages, tmp_dir):\n    \"\"\"Download .deb files in parallel.\"\"\"\n    os.makedirs(tmp_dir, exist_ok=True)\n\n    tasks = []\n    timeout = aiohttp.ClientTimeout(total=60)\n    async with aiohttp.ClientSession(timeout=timeout) as session:\n        for pkg, info in packages.items():\n            filename = info.get(\"Filename\")\n            if filename:\n                url = f\"{mirror}/{filename}\"\n                dest_path = os.path.join(tmp_dir, os.path.basename(filename))\n                tasks.append(asyncio.create_task(download_file(session, url, dest_path, checksum=info.get(\"SHA256\"))))\n\n        await asyncio.gather(*tasks)\n\nasync def download_package_index_parallel(mirror, arch, suites, check_sig, keyring):","sourceCodeStart":29,"sourceCodeEnd":65,"githubUrl":"https://github.com/dotnet/aspnetcore/blob/294cab2f9b2e03af6b953820c7ab497c3c8b7ad9/eng/common/cross/install-debs.py#L29-L65","documentation":"download_file retries up to max_retries (default 3) but only catches asyncio.CancelledError, asyncio.TimeoutError, and aiohttp.ClientError. If all attempts fail with one of those exception types, the loop exhausts and raises a final Exception naming the URL and attempt count. This is the terminal failure for transient network problems that did not self-heal within the retry budget.","triggerScenarios":"Sustained DNS failure, connection refused, TLS handshake error, or read timeout that persists across all 3 attempts spaced retry_delay (default 2s) apart. Each failure prints 'Error downloading ... Retrying...' and the loop increments attempt. After the final failure, this exception is raised and aborts asyncio.gather.","commonSituations":"No network egress from the build host to the mirror; DNS misconfigured in the build container; mirror is down or unreachable; firewall blocking outbound HTTPS; proxy environment variables (HTTP_PROXY/HTTPS_PROXY) point at a dead proxy.","solutions":["Verify outbound connectivity from the host: curl -I <mirror-url> before re-running.","Set correct HTTP_PROXY/HTTPS_PROXY/NO_PROXY environment variables if behind a corporate proxy, or unset them if they are wrong.","Increase robustness by passing a larger max_retries/retry_delay to download_file (requires code change) for flaky links.","Switch to a mirror reachable from the build environment (e.g., a locally mirrored copy or snapshot.debian.org).","If building in CI, confirm the runner has network access and is not sandboxed to a package allowlist."],"exampleFix":"# before — running with no proxy in a corporate network\nHTTPS_PROXY=http://dead-proxy:8080 python3 install-debs.py ...\n# Failed to download after 3 attempts\n\n# after\nunset HTTPS_PROXY HTTP_PROXY\npython3 install-debs.py --mirror http://deb.debian.org/debian ...","handlingStrategy":"retry","validationCode":"# Verify connectivity before invoking install-debs\nimport socket\nhost = urllib.parse.urlparse(mirror).hostname\ntry:\n    socket.create_connection((host, 443), timeout=5).close()\nexcept OSError:\n    raise SystemExit(f'No route to mirror {host}; fix network/proxy first')","typeGuard":null,"tryCatchPattern":"# download_file only catches transient errors; wrap the whole gather for resilience\nimport asyncio\nfor attempt in range(3):\n    try:\n        await asyncio.gather(*tasks)\n        break\n    except Exception as e:\n        if 'after' in str(e) and 'attempts' in str(e):\n            print(f'All retries exhausted ({attempt+1}/3); switching mirror or backoff')\n            await asyncio.sleep(2 ** attempt)\n            continue\n        raise","preventionTips":["Confirm outbound connectivity (DNS, proxy, firewall) before running.","Set HTTP_PROXY/HTTPS_PROXY correctly in corporate networks, or unset them when wrong.","For flaky links, raise max_retries/retry_delay in download_file.","Use a local mirror to remove network as a variable."],"tags":["python","debian","network","retry","timeout","install-debs","rootfs"],"analyzedSha":"294cab2f9b2e03af6b953820c7ab497c3c8b7ad9","analyzedAt":"2026-08-06T20:08:02.189Z","schemaVersion":2},"datasetVersion":"2026-08-06T23:17:07.152Z"}