apache/dolphinscheduler · error · FileAlreadyExistsException

directory: ${cosKey} already exists

Error message

directory: ${cosKey} already exists

What it means

CosStorageOperator.createStorageDir creates a directory in Tencent COS by writing a zero-length object at the transformed key. If an object already exists at that key, it throws FileAlreadyExistsException instead of overwriting. COS directories are only emulated via key prefixes, so any pre-existing object at the key collides.

Source

Thrown at dolphinscheduler-storage-plugin/dolphinscheduler-storage-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java:127

    public String getStorageBaseDirectory() {
        // All directory should end with File.separator
        if (resourceBaseAbsolutePath.startsWith(File.separator)) {
            String warnMessage =
                    String.format("%s -> %s should not start with %s in tencent cos",
                            StorageConstants.RESOURCE_UPLOAD_PATH,
                            resourceBaseAbsolutePath, File.separator);
            log.warn(warnMessage);
            return resourceBaseAbsolutePath.substring(1);
        }
        return resourceBaseAbsolutePath;
    }

    @SneakyThrows
    @Override
    public void createStorageDir(String directoryAbsolutePath) {
        String cosKey = transformAbsolutePathToCOSKey(directoryAbsolutePath);
        if (cosClient.doesObjectExist(bucketName, cosKey)) {
            throw new FileAlreadyExistsException("directory: " + cosKey + " already exists");
        }
        ObjectMetadata metadata = new ObjectMetadata();
        metadata.setContentLength(0L);
        InputStream emptyContent = new ByteArrayInputStream(new byte[0]);
        PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, cosKey, emptyContent, metadata);
        cosClient.putObject(putObjectRequest);
    }

    @SneakyThrows
    @Override
    public void download(String srcFilePath, String dstFilePath, boolean overwrite) {
        String cosKey = transformAbsolutePathToCOSKey(srcFilePath);
        Path dsTempFolder = Paths.get(FileUtils.DATA_BASEDIR).normalize().toAbsolutePath();
        Path fileDownloadPathNormalized = dsTempFolder.resolve(dstFilePath).normalize().toAbsolutePath();
        if (!fileDownloadPathNormalized.startsWith(dsTempFolder)) {
            // if the destination file path is NOT in DS temp folder (e.g., '/tmp/dolphinscheduler'),
            // an IllegalArgumentException should be thrown.
            throw new IllegalArgumentException("failed to download to " + fileDownloadPathNormalized);

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Check existence first with exists()/fetchFileList and skip or use a unique directory name
2. Delete the existing object (cosClient.deleteObject) if it is safe to overwrite, then retry createStorageDir
3. Use a timestamped or tenant-scoped directory path to avoid collisions

Example fix

// before
storageOperator.createStorageDir(resourcePath);
// after
if (!storageOperator.exists(resourcePath)) {
    storageOperator.createStorageDir(resourcePath);
}

if (cosStorageOperator.exists(cosKey)) { throw new IllegalStateException(cosKey + " exists; choose another directory"); }

try {
    operator.createStorageDir(dir);
} catch (FileAlreadyExistsException e) {
    log.info("Directory already present, reusing: {}", dir);
}

When it happens

Trigger: Calling createStorageDir(directoryAbsolutePath) when cosClient.doesObjectExist(bucketName, cosKey) returns true — i.e., a prior mkdir, a file uploaded under the same path, or a leftover marker object.

Common situations: Re-running a workflow that previously created the same resource directory; uploading a resource file that occupies the key the directory wants; stale COS objects after tenant/resource renames.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

• file: ${dstPath} already exists
• failed to download to ${fileDownloadPathNormalized}
• directory: ${directoryAbsolutePath} already exists
• file: ${dstPath} already exists
• Directory already exists: ${directoryAbsolutePath}

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/69af945d87535a35. Report an issue: GitHub.