apache/flink · error · UnsupportedOperationException

Bulk copy not enabled. Set s3.bulk-copy.enabled=true

Error message

Bulk copy not enabled. Set s3.bulk-copy.enabled=true

What it means

NativeS3FileSystem.copyFiles requires the bulk-copy helper, which is only constructed when s3.bulk-copy.enabled=true in the filesystem factory configuration. When the flag is off, bulkCopyHelper is null and any copyFiles call throws UnsupportedOperationException pointing at the missing option.

Source

Thrown at flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystem.java:527

        return StringUtils.generateRandomAlphanumericString(
                ThreadLocalRandom.current(), entropyLength);
    }

    @Override
    public boolean canCopyPaths(Path source, Path destination) {
        return bulkCopyHelper != null
                && NativeS3BulkCopyHelper.isSupportedS3Scheme(source)
                && NativeS3BulkCopyHelper.isSupportedLocalScheme(destination);
    }

    @Override
    public void copyFiles(
            List<CopyRequest> requests,
            org.apache.flink.core.fs.ICloseableRegistry closeableRegistry)
            throws IOException {
        checkNotClosed();
        if (bulkCopyHelper == null) {
            throw new UnsupportedOperationException(
                    "Bulk copy not enabled. Set s3.bulk-copy.enabled=true");
        }
        bulkCopyHelper.copyFiles(requests, closeableRegistry);
    }

    @Override
    public RecoverableWriter createRecoverableWriter() throws IOException {
        checkNotClosed();
        if (s3AccessHelper == null) {
            throw new UnsupportedOperationException("Recoverable writer not available");
        }
        return NativeS3RecoverableWriter.writer(
                s3AccessHelper, localTmpDir, s3uploadPartSize, maxConcurrentUploadsPerStream);
    }

    @Override
    public CompletableFuture<Void> closeAsync() {
        if (!closed.compareAndSet(false, true)) {

View on GitHub (pinned to 2f3c205e92)

Solutions

Set s3.bulk-copy.enabled: true in flink-conf and restart/re-create the filesystem.
Verify the option key with NativeS3FileSystemFactory (BULK_COPY_ENABLED) so spelling matches your Flink version.
Alternatively avoid the copyFiles path (e.g. disable local recovery) if bulk copy is not desired.

Example fix

# before
# no bulk copy config

# after
fs.s3.bulk-copy.enabled: true
# (confirm exact key prefix: s3.bulk-copy.enabled for your version)

Defensive patterns

Strategy: validation

Validate before calling

// check capability before relying on bulk copy
if (!(s3Fs instanceof PathsCopyingFileSystem)
        || !((PathsCopyingFileSystem) s3Fs).supportsCopyFiles(source, destination)) {
    throw new UnsupportedOperationException("Bulk copy unavailable; set s3.bulk-copy.enabled=true");
}

Type guard

static boolean bulkCopyAvailable(FileSystem fs) {
    return fs instanceof PathsCopyingFileSystem
            && ((PathsCopyingFileSystem) fs).supportsCopyFiles(
                    new Path("s3://probe/obj"), new Path("file:///tmp/probe"));
}

Try / catch

try {
    ((PathsCopyingFileSystem) fs).copyFiles(requests, registry);
} catch (UnsupportedOperationException e) {
    if (e.getMessage().contains("s3.bulk-copy.enabled")) {
        // enable the flag in config, recreate the filesystem, then retry
    } else {
        throw e;
    }
}

Prevention

Set s3.bulk-copy.enabled=true whenever a feature documents bulk-copy dependency (e.g. S3 state download).
Prefer the filesystem's supportsCopyFiles capability check over catching exceptions.
Keep S3 plugin jars in sync with the Flink version so new options are recognized.

When it happens

Trigger: Calling PathsCopyingFileSystem.copyFiles (e.g. via Flink's staging/recovery path for RocksDB incremental checkpoints or savepoints stored on S3) on an S3 filesystem created without s3.bulk-copy.enabled=true.

Common situations: Enabling a feature that relies on bulk copy (local recovery, download of state from S3) without turning on the new bulk-copy optimization; upgrading Flink where the flag defaults to false.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/efed47ffb127c273. Report an issue: GitHub.