apache/dolphinscheduler · error · FileAlreadyExistsException

Directory already exists: ${directoryAbsolutePath}

Error message

Directory already exists: ${directoryAbsolutePath}

What it means

HdfsStorageOperator.createStorageDir creates a directory on HDFS via fs.mkdirs. If the HDFS path already exists it throws FileAlreadyExistsException instead of silently succeeding, so callers can detect the collision.

Source

Thrown at dolphinscheduler-storage-plugin/dolphinscheduler-storage-hdfs/src/main/java/org/apache/dolphinscheduler/plugin/storage/hdfs/HdfsStorageOperator.java:166

    @Override
    public List<String> fetchFileContent(String hdfsFilePath, int skipLineNums, int limit) {
        try (
                FSDataInputStream in = fs.open(new Path(hdfsFilePath));
                InputStreamReader inputStreamReader = new InputStreamReader(in, StandardCharsets.UTF_8);
                BufferedReader br = new BufferedReader(inputStreamReader)) {
            return br.lines()
                    .skip(skipLineNums)
                    .limit(limit)
                    .collect(Collectors.toList());
        }
    }

    @Override
    @SneakyThrows
    public void createStorageDir(String directoryAbsolutePath) {
        Path path = new Path(directoryAbsolutePath);
        if (fs.exists(path)) {
            throw new FileAlreadyExistsException("Directory already exists: " + directoryAbsolutePath);
        }
        fs.mkdirs(new Path(directoryAbsolutePath));
    }

    @SneakyThrows
    @Override
    public void download(String srcHdfsFilePath, String dstFile, boolean overwrite) {
        Path srcPath = new Path(srcHdfsFilePath);
        File dstPath = new File(dstFile);

        if (dstPath.exists()) {
            if (dstPath.isFile()) {
                if (overwrite) {
                    Files.delete(dstPath.toPath());
                }
            } else {
                log.error("destination file must be a file");
            }

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Check fs.exists / operator.exists first and skip creation if present
2. Delete the existing path (hdfs dfs -rm -r) if overwrite is safe, then retry
3. Use a unique path per execution (workflow instance id, timestamp)

Example fix

// before
hdfs.createStorageDir(uploadPath);
// after
if (!hdfs.exists(uploadPath)) {
    hdfs.createStorageDir(uploadPath);
}

if (hdfs.exists(hdfsPath)) { /* skip or choose a new path */ }

try {
    hdfs.createStorageDir(path);
} catch (FileAlreadyExistsException e) {
    log.info("HDFS directory exists, reusing: {}", path);
}

When it happens

Trigger: Calling createStorageDir(directoryAbsolutePath) when fs.exists(new Path(directoryAbsolutePath)) is true — a previous mkdir, a resource file already at that path, or a rerun of the same workflow.

Common situations: Re-running deployments that create the same resource directory; the path being occupied by a file rather than a directory; shared tenant paths reused across workflows.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

• directory: ${cosKey} already exists
• file: ${dstPath} already exists
• directory: ${directoryAbsolutePath} already exists
• file: ${dstPath} already exists
• Download resource file: %s error

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/2105eccc413778c2. Report an issue: GitHub.