{"record":{"id":"8442c592ac822df3","repo":"apache/flink","slug":"bulk-copy-failed","errorCode":null,"errorMessage":"Bulk copy failed","messagePattern":"Bulk copy failed","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3BulkCopyHelper.java","lineNumber":387,"sourceCode":"            Thread.currentThread().interrupt();\n            throw new IOException(\"Bulk copy interrupted\", e);\n        } catch (ExecutionException e) {\n            Throwable cause = e.getCause();\n            ExceptionUtils.rethrowIfFatalError(cause);\n            if (isConnectionPoolExhausted(cause)) {\n                throw new IOException(\n                        String.format(\n                                \"S3 connection pool exhausted during bulk copy. \"\n                                        + \"The configured connection pool size (%d) could not serve \"\n                                        + \"the concurrent download requests (%d). \"\n                                        + \"Consider reducing '%s' or increasing '%s'.\",\n                                maxConnections,\n                                maxConcurrentCopies,\n                                NativeS3FileSystemFactory.BULK_COPY_MAX_CONCURRENT.key(),\n                                NativeS3FileSystemFactory.MAX_CONNECTIONS.key()),\n                        cause);\n            }\n            throw new IOException(\"Bulk copy failed\", cause);\n        }\n    }\n\n    static boolean isSupportedS3Scheme(org.apache.flink.core.fs.Path path) {\n        String scheme = path.toUri().getScheme();\n        return \"s3\".equalsIgnoreCase(scheme) || \"s3a\".equalsIgnoreCase(scheme);\n    }\n\n    static boolean isSupportedLocalScheme(org.apache.flink.core.fs.Path path) {\n        String scheme = path.toUri().getScheme();\n        return scheme == null || \"file\".equalsIgnoreCase(scheme);\n    }\n\n    private static void abortAndClose(ResponseInputStream<GetObjectResponse> stream) {\n        try {\n            stream.abort();\n        } catch (RuntimeException e) {\n            LOG.debug(\"Error aborting S3 response stream during bulk-copy cancellation\", e);","sourceCodeStart":369,"sourceCodeEnd":405,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3BulkCopyHelper.java#L369-L405","documentation":"Generic wrapper IOException thrown by NativeS3BulkCopyHelper when the bulk copy batch failed with a non-interruption, non-pool-exhaustion error. The real cause (S3Exception, NoSuchKey, credentials error, disk full, etc.) is attached as the cause and should be inspected first.","triggerScenarios":"Any ExecutionException from a copy future that is not connection-pool exhaustion: 404 NoSuchKey on a source object, 403 access denied, local disk I/O failure writing the destination, SDK client shutdown, or network errors.","commonSituations":"Source objects deleted between listing and download (race with compaction/cleanup), missing s3:GetObject permissions, local tmp directory full or read-only, or the S3 client closed while copies were pending.","solutions":["Inspect getCause() of the IOException to identify the underlying failure; the message 'Bulk copy failed' alone is intentionally generic.","Fix the root cause: restore access to the object, grant permissions, free disk space, or re-create the client.","Re-run the copy; it is idempotent per file if destinations are overwritten."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    s3Fs.copyFiles(requests, registry);\n} catch (IOException e) {\n    Throwable cause = e.getCause();\n    if (cause instanceof NoSuchKeyException) {\n        // source object vanished: log and skip / re-list\n    } else if (cause instanceof S3Exception && ((S3Exception) cause).statusCode() >= 500) {\n        // transient: retry with backoff after cleaning partial local files\n    } else {\n        throw e;\n    }\n}","preventionTips":["Always inspect the cause chain — 'Bulk copy failed' is intentionally generic.","Clean partially written local destinations before retrying.","Ensure source objects are not deleted concurrently by compaction/cleanup jobs."],"tags":["s3","bulk-copy","wrapper-exception","diagnostics"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}