apache/dolphinscheduler · error · UnsupportedOperationException

S3 does not support copying directories.

Error message

S3 does not support copying directories.

What it means

S3StorageOperator.copy throws UnsupportedOperationException when the source path resolves to a directory. The S3 plugin only implements single-object CopyObject; recursive directory copy is not supported and is rejected up front.

Source

Thrown at dolphinscheduler-storage-plugin/dolphinscheduler-storage-s3/src/main/java/org/apache/dolphinscheduler/plugin/storage/s3/S3StorageOperator.java:165

            return;
        }
        if (recursive) {
            List<StorageEntity> storageEntities = listStorageEntityRecursively(absolutePath);
            for (StorageEntity storageEntity : storageEntities) {
                s3Client.deleteObject(bucketName, transformAbsolutePathToS3Key(storageEntity.getFullName()));
            }
        }
        s3Client.deleteObject(bucketName, absolutePath);
    }

    @Override
    public void copy(String srcPath, String dstPath, boolean deleteSource, boolean overwrite) {
        srcPath = transformAbsolutePathToS3Key(srcPath);
        dstPath = transformAbsolutePathToS3Key(dstPath);

        ResourceMetadata resourceMetaData = getResourceMetaData(srcPath);
        if (resourceMetaData.isDirectory()) {
            throw new UnsupportedOperationException("S3 does not support copying directories.");
        }
        s3Client.copyObject(bucketName, srcPath, bucketName, dstPath);
        if (deleteSource) {
            s3Client.deleteObject(bucketName, srcPath);
        }
    }

    @SneakyThrows
    @Override
    public void upload(String srcFile, String dstPath, boolean deleteSource, boolean overwrite) {
        dstPath = transformAbsolutePathToS3Key(dstPath);

        if (s3Client.doesObjectExist(bucketName, dstPath)) {
            if (overwrite) {
                s3Client.deleteObject(bucketName, dstPath);
            } else {
                throw new FileAlreadyExistsException("The file " + dstPath + " already exists in the bucket "
                        + bucketName + " and overwrite is not allowed.");

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Copy individual files: enumerate directory entries (listStorageEntity) and call copy per file, then delete the source files
2. Perform a server-side bulk operation outside the plugin (S3 Batch Operations) for large folders
3. Use a storage operator that supports directory copy (HDFS/local) if folder semantics are required

Example fix

// before
storageOperator.copy("/resources/folder", "/archive/folder", true, false);
// after
for (StorageEntity entity : storageOperator.listStorageEntity("/resources/folder")) {
    storageOperator.copy(entity.getPath(),
        entity.getPath().replaceFirst("^/resources", "/archive"), true, false);
}

ResourceMetadata meta = s3StorageOperator.getResourceMetaData(srcPath);
if (meta.isDirectory()) {
    throw new IllegalArgumentException("Use per-file copy for directories: " + srcPath);
}
s3StorageOperator.copy(srcPath, dstPath, deleteSource, overwrite);

try {
    s3StorageOperator.copy(src, dst, deleteSource, overwrite);
} catch (UnsupportedOperationException e) {
    log.warn("Directory copy unsupported by S3 plugin; falling back to per-file copy");
    copyDirectoryFileByFile(src, dst, deleteSource, overwrite);
}

When it happens

Trigger: Calling copy(srcPath, dstPath, deleteSource, overwrite) where getResourceMetaData(srcPath).isDirectory() is true — the source key is a directory marker or has children.

Common situations: Trying to move/rename a whole resource folder via the storage API (e.g. renaming a user's resource directory) or porting code that worked against local/HDFS storage operators.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• The directory ${directoryAbsolutePath} already exists in the
• The file ${dstPath} already exists in the bucket ${bucketNam
• Download resource file: %s error
• Unsupported executeType: {0}
• Update the resource file from content: {fileAbsolutePath} fa

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/8786df2ea755ea2a. Report an issue: GitHub.