apache/hadoop · error · FileAlreadyExistsException

Failed to rename since the dest path %s already exists.

Error message

Failed to rename since the dest path %s already exists.

What it means

rename() in RawFileSystem throws FileAlreadyExistsException when the final destination already exists. Hadoop's atomic-rename contract forbids overwriting: if dest is a file, or dest is a directory and dest/srcName already exists, the rename is rejected.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java:316

    // 2. No need to check the dest path because renaming itself is allowed.
    if (src.equals(finalDstPath)) {
      return finalDstPath;
    }

    // 3. Ensure the source path cannot be the ancestor of destination path.
    if (RawFSUtils.inSubtree(src, finalDstPath)) {
      throw new IOException(String.format("Failed to rename since it is prohibited to " +
          "rename dest path %s under src path %s", finalDstPath, src));
    }

    // 4. Ensure the destination path doesn't exist.
    FileStatus finalDstStatus = destStatus;
    if (destStatus != null && destStatus.isDirectory()) {
      finalDstStatus = getFileStatusOrNull(finalDstPath);
    }
    if (finalDstStatus != null) {
      throw new FileAlreadyExistsException(
          String.format("Failed to rename since the dest path %s already exists.", finalDstPath));
    } else {
      return finalDstPath;
    }
  }

  private FileStatus checkAndGetSrcStatus(Path src) throws IOException {
    // throw FileNotFoundException if src not found.
    FileStatus srcStatus = innerFileStatus(src);

    if (src.isRoot()) {
      throw new IOException(String.format("Cannot rename the root directory %s to another name",
          src));
    }
    return srcStatus;
  }

  @Override

View on GitHub (pinned to 2add963021)

Solutions

1. Delete or move the colliding destination object first (fs.delete(finalDst, false)) when overwrite semantics are acceptable
2. Guard the rename: if (!fs.exists(finalDst)) fs.rename(src, dst), else pick a unique destination name
3. Clean the destination directory between runs instead of reusing it

Example fix

// before
fs.rename(src, dstDir); // dstDir/srcName already exists -> FileAlreadyExistsException

// after
Path finalDst = fs.getFileStatus(dstDir).isDirectory()
    ? new Path(dstDir, src.getName()) : dstDir;
if (fs.exists(finalDst)) { fs.delete(finalDst, false); }
fs.rename(src, dstDir);

Path finalDst = fs.exists(dst) && fs.getFileStatus(dst).isDirectory()
    ? new Path(dst, src.getName()) : dst;
if (fs.exists(finalDst)) {
  fs.delete(finalDst, false); // or choose a unique name
}
boolean renamed = fs.rename(src, dst);

try { fs.rename(src, dst); }
catch (FileAlreadyExistsException e) {
  // destination collision: delete/rename target and retry once
}

When it happens

Trigger: rename(src, dst) where dst exists as a file; or dst exists as a directory and new Path(dst, src.getName()) already exists (the 'rename into directory' adjustment), as detected via getFileStatusOrNull(finalDstPath).

Common situations: Idempotent job re-runs that 'move' results into a directory already populated by a previous run; merge scripts that collapse two trees into one; leftover destination objects from an aborted attempt.

Related errors

• File: %s already exists
• {} is a directory
• {} already exists
• Failed to rename since it is prohibited to rename dest path
• Cannot rename the root directory %s to another name

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b762acecd865511c. Report an issue: GitHub.