{"record":{"id":"f5cc689729b32545","repo":"alibaba/canal","slug":"download-failed-url-statuscode","errorCode":null,"errorMessage":"download failed , url:{} , statusCode:{}","messagePattern":"download failed , url:(.+?) , statusCode:(.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/rds/BinlogDownloadQueue.java","lineNumber":203,"sourceCode":"                    .register(\"https\", new SSLConnectionSocketFactory(sslContext, NoopHostnameVerifier.INSTANCE))\n                    .build()))\n                .build();\n        } else {\n            httpClient = HttpClientBuilder.create().setMaxConnPerRoute(50).setMaxConnTotal(100).build();\n        }\n\n        try {\n            HttpGet httpGet = new HttpGet(downloadLink);\n            RequestConfig requestConfig = RequestConfig.custom()\n                .setConnectTimeout(TIMEOUT)\n                .setConnectionRequestTimeout(TIMEOUT)\n                .setSocketTimeout(TIMEOUT)\n                .build();\n            httpGet.setConfig(requestConfig);\n            HttpResponse response = httpClient.execute(httpGet);\n            int statusCode = response.getStatusLine().getStatusCode();\n            if (statusCode != HttpResponseStatus.OK.code()) {\n                throw new RuntimeException(\"download failed , url:\" + downloadLink + \" , statusCode:\" + statusCode);\n            }\n            saveFile(new File(destDir), \"mysql-bin.\" + fileName, response);\n        } finally {\n            httpClient.close();\n        }\n    }\n\n    private static void saveFile(File parentFile, String fileName, HttpResponse response) throws IOException {\n        InputStream is = response.getEntity().getContent();\n        boolean isChunked = response.getEntity().isChunked();\n        Header contentLengthHeader = response.getFirstHeader(\"Content-Length\");\n        long totalSize = (isChunked || contentLengthHeader == null) ? 0 : Long.parseLong(contentLengthHeader.getValue());\n        if (response.getFirstHeader(\"Content-Disposition\") != null) {\n            fileName = response.getFirstHeader(\"Content-Disposition\").getValue();\n            fileName = StringUtils.substringAfter(fileName, \"filename=\");\n        }\n        boolean isTar = StringUtils.endsWith(fileName, \".tar\");\n        FileUtils.forceMkdir(parentFile);","sourceCodeStart":185,"sourceCodeEnd":221,"githubUrl":"https://github.com/alibaba/canal/blob/87be50e87686a3e8af08c368d0e1ffd1f59eb04a/parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/rds/BinlogDownloadQueue.java#L185-L221","documentation":"Thrown as RuntimeException from the RDS binlog download path when the HTTP GET to the RDS binlog download link returns a non-200 status code. The exception includes the download URL and the HTTP status code. This is the per-file download failure — the RDS API provided a pre-signed download URL, but fetching the actual binlog file content failed.","triggerScenarios":"httpClient.execute(httpGet) returns an HttpResponse whose status code is not 200 (HttpResponseStatus.OK.code()). Common non-200 codes: 403 (expired or invalid pre-signed URL), 404 (binlog file removed), 408/504 (timeout), 429 (rate limited), 500/502/503 (RDS storage backend error).","commonSituations":"The pre-signed download URL expired before the download started (RDS URLs have a short TTL). RDS rate-limited the download request. The RDS storage backend had a transient error. Network issues between Canal and the RDS OSS/storage endpoint. The binlog file was purged between the API listing and the actual download.","solutions":["Check the status code: 403 = expired URL (re-request from RDS API), 404 = file gone, 429 = rate limit, 5xx = server error.","Increase the TIMEOUT constant in BinlogDownloadQueue if downloads are timing out on large binlog files.","Implement retry logic with exponential backoff around the download call for transient 5xx/429 errors.","If 403 persists, verify the AccessKey/SecretKey are valid and not expired."],"exampleFix":"// before — single attempt, no retry\nBinlogFile binlogFile = downloadQueue.tryOne();\n\n// after — retry with backoff for transient failures\nint maxRetries = 3;\nfor (int i = 0; i < maxRetries; i++) {\n    try {\n        BinlogFile binlogFile = downloadQueue.tryOne();\n        break;\n    } catch (RuntimeException e) {\n        if (i == maxRetries - 1) throw e;\n        Thread.sleep((long) Math.pow(2, i) * 1000);\n    }\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"int maxRetries = 3;\nfor (int attempt = 0; attempt < maxRetries; attempt++) {\n    try {\n        BinlogFile file = downloadQueue.tryOne();\n        break;\n    } catch (RuntimeException e) {\n        if (e.getMessage().contains(\"download failed\")) {\n            int statusCode = extractStatusCode(e.getMessage());\n            if (statusCode == 403) {\n                // URL expired — re-request from RDS API\n                refreshDownloadLinks();\n            } else if (statusCode >= 500 && attempt < maxRetries - 1) {\n                Thread.sleep((long) Math.pow(2, attempt) * 1000);\n                continue;\n            }\n        }\n        throw e;\n    }\n}","preventionTips":["Pre-signed RDS download URLs expire quickly — download promptly after obtaining them.","Increase the TIMEOUT constant for large binlog files that take longer to download.","Implement exponential backoff for 5xx and 429 HTTP responses."],"tags":["rds","alibaba-cloud","http","download","binlog-download"],"backgroundTag":null,"analyzedSha":"87be50e87686a3e8af08c368d0e1ffd1f59eb04a","analyzedAt":"2026-08-14T04:30:11.918Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}