apache/dolphinscheduler · warning · FileAlreadyExistsException

The directory ${directoryAbsolutePath} already exists in the

Error message

The directory ${directoryAbsolutePath} already exists in the bucket ${bucketName}

What it means

S3StorageOperator.createStorageDir throws FileAlreadyExistsException when an object already exists at the transformed S3 key in the given bucket. S3 'directories' are zero-byte key markers, so an existing object at that key prevents directory creation.

Source

Thrown at dolphinscheduler-storage-plugin/dolphinscheduler-storage-s3/src/main/java/org/apache/dolphinscheduler/plugin/storage/s3/S3StorageOperator.java:100

        if (resourceBaseAbsolutePath.startsWith("/")) {
            log.warn("{} -> {} should not start with / in s3", StorageConstants.RESOURCE_UPLOAD_PATH,
                    resourceBaseAbsolutePath);
            return resourceBaseAbsolutePath.substring(1);
        }
        return resourceBaseAbsolutePath;
    }

    @Override
    public void close() throws IOException {
        s3Client.shutdown();
    }

    @SneakyThrows
    @Override
    public void createStorageDir(String directoryAbsolutePath) {
        directoryAbsolutePath = transformAbsolutePathToS3Key(directoryAbsolutePath);
        if (s3Client.doesObjectExist(bucketName, directoryAbsolutePath)) {
            throw new FileAlreadyExistsException(
                    "The directory " + directoryAbsolutePath + " already exists in the bucket " + bucketName);
        }
        ObjectMetadata metadata = new ObjectMetadata();
        metadata.setContentLength(0);
        InputStream emptyContent = new ByteArrayInputStream(new byte[0]);
        PutObjectRequest putObjectRequest =
                new PutObjectRequest(bucketName, directoryAbsolutePath, emptyContent, metadata);
        s3Client.putObject(putObjectRequest);
    }

    @SneakyThrows
    @Override
    public void download(String srcFilePath,
                         String dstFilePath,
                         boolean overwrite) {
        srcFilePath = transformAbsolutePathToS3Key(srcFilePath);
        File dstFile = new File(dstFilePath);
        if (dstFile.isDirectory()) {

View on GitHub (pinned to 02eac45a1b)

Solutions

Check existence (existsStorageDir) before creating, or delete the object first
Catch FileAlreadyExistsException and treat it as success for idempotent setup
Generate unique directory keys per run

Example fix

// before
s3StorageOperator.createStorageDir(resourceDir);
// after
if (!s3StorageOperator.existsStorageDir(resourceDir)) {
    s3StorageOperator.createStorageDir(resourceDir);
}

Defensive patterns

Strategy: validation

Validate before calling

if (!s3StorageOperator.existsStorageDir(directoryAbsolutePath)) {
    s3StorageOperator.createStorageDir(directoryAbsolutePath);
}

Try / catch

try {
    s3StorageOperator.createStorageDir(key);
} catch (FileAlreadyExistsException e) {
    log.info("S3 directory {} already exists in bucket, skipping", key);
}

Prevention

Make S3 directory creation idempotent (check first or ignore the exception)
Never upload files to keys you also use as directory markers
Use execution-scoped key prefixes to prevent cross-run collisions

When it happens

Trigger: Calling createStorageDir(directoryAbsolutePath) when s3Client.doesObjectExist(bucketName, s3Key) is true — the marker or a file at that exact key already exists.

Common situations: Re-running workflows that provision the same resource directory; a previous task uploaded a file using the key now intended as a directory; concurrent task instances creating the same directory.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/ee653dbbc1960d52. Report an issue: GitHub.