apache/hadoop · error · FileAlreadyExistsException

Destination path {dst} already exists

Error message

Destination path {dst} already exists

What it means

rename() never overwrites: after resolving the destination — including the rule that a directory dst becomes dst/srcName — it probes existence and throws FileAlreadyExistsException("Destination path <dst> already exists"). This mirrors HDFS rename semantics where the target must not be clobbered.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FTPFileSystem.java:685

   * @return
   * @throws IOException
   */
  @SuppressWarnings("deprecation")
  private boolean rename(FTPClient client, Path src, Path dst)
      throws IOException {
    Path workDir = new Path(client.printWorkingDirectory());
    Path absoluteSrc = makeAbsolute(workDir, src);
    Path absoluteDst = makeAbsolute(workDir, dst);
    if (!exists(client, absoluteSrc)) {
      throw new FileNotFoundException("Source path " + src + " does not exist");
    }
    if (isDirectory(absoluteDst)) {
      // destination is a directory: rename goes underneath it with the
      // source name
      absoluteDst = new Path(absoluteDst, absoluteSrc.getName());
    }
    if (exists(client, absoluteDst)) {
      throw new FileAlreadyExistsException("Destination path " + dst
          + " already exists");
    }
    URI parentSrc = absoluteSrc.getParent().toUri();
    URI parentDst = absoluteDst.getParent().toUri();
    if (isParentOf(absoluteSrc, absoluteDst)) {
      throw new IOException("Cannot rename " + absoluteSrc + " under itself"
      + " : "+ absoluteDst);
    }

    if (!parentSrc.toString().equals(parentDst.toString())) {
      throw new IOException("Cannot rename source: " + absoluteSrc
          + " to " + absoluteDst
          + " -"+ E_SAME_DIRECTORY_ONLY);
    }
    String from = absoluteSrc.getName();
    String to = absoluteDst.getName();
    client.changeWorkingDirectory(parentSrc.getPath());
    boolean renamed = client.rename(from, to);

View on GitHub (pinned to 2add963021)

Solutions

1. Choose a unique destination (timestamp/UUID suffix) or delete/backup the target first when overwrite is intended
2. Remember the expansion rule: when dst is a directory the real checked path is dst/srcName — check that path
3. Re-check fs.exists(dst) right before rename and accept the small race window with a retry
4. For atomic publish patterns, rename into a fresh directory instead of over existing files

Example fix

// before
fs.rename(src, dst);  // FileAlreadyExistsException

// after
if (fs.exists(dst)) {
  Path backup = new Path(dst.getParent(), dst.getName() + "." + System.currentTimeMillis());
  fs.rename(dst, backup);   // or fs.delete(dst, false) if overwrite is truly wanted
}
fs.rename(src, dst);

// pick a non-colliding destination (remember: dst-dir expands to dst/srcName)
Path target = fs.exists(dst) || fs.exists(new Path(dst, src.getName()))
    ? new Path(dst.getParent(), dst.getName() + "." + System.currentTimeMillis())
    : dst;
fs.rename(src, target);

catch FileAlreadyExistsException specifically and respond by choosing a new destination or removing the target — do not retry the same rename unchanged.

When it happens

Trigger: rename(a, b) with b already present; rename(a, dir) where dir/a already exists because of the dst-directory expansion rule; a racing writer creating the target between your exists() check and the rename.

Common situations: Idempotent job reruns writing to the same output name; ingest pipelines that "move into place" without unique names; developers assuming POSIX mv overwrite semantics.

Related errors

• rename destination ${dst} already exists.
• Source path {src} does not exist
• Cannot rename {absoluteSrc} under itself : {absoluteDst}
• Cannot rename source: {absoluteSrc} to {absoluteDst} -only s
• Source path and dest path refer the same file or directory

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f3555eeb79f03adc. Report an issue: GitHub.