apache/hadoop · error · FileAlreadyExistsException

Rename destination {dst} already exists.

Error message

Rename destination {dst} already exists.

What it means

Non-overwrite rename is POSIX-like: if the destination already exists, AbstractFileSystem.renameInternal throws FileAlreadyExistsException before touching anything. FileContext.rename(src, dst) without options maps to Rename.NONE, so any pre-existing file, directory, or symlink at dst triggers it.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java:876

    } catch (IOException e) {
      dstStatus = null;
    }
    if (dstStatus != null) {
      if (dst.equals(src)) {
        throw new FileAlreadyExistsException(
            "The source "+src+" and destination "+dst+" are the same");
      }
      if (srcStatus.isSymlink() && dst.equals(srcStatus.getSymlink())) {
        throw new FileAlreadyExistsException(
            "Cannot rename symlink "+src+" to its target "+dst);
      }
      // It's OK to rename a file to a symlink and vice versa
      if (srcStatus.isDirectory() != dstStatus.isDirectory()) {
        throw new IOException("Source " + src + " and destination " + dst
            + " must both be directories");
      }
      if (!overwrite) {
        throw new FileAlreadyExistsException("Rename destination " + dst
            + " already exists.");
      }
      // Delete the destination that is a file or an empty directory
      if (dstStatus.isDirectory()) {
        RemoteIterator<FileStatus> list = listStatusIterator(dst);
        if (list != null && list.hasNext()) {
          throw new IOException(
              "Rename cannot overwrite non empty destination directory " + dst);
        }
      }
      delete(dst, false);
    } else {
      final Path parent = dst.getParent();
      final FileStatus parentStatus = getFileStatus(parent);
      if (parentStatus.isFile()) {
        throw new ParentNotDirectoryException("Rename destination parent "
            + parent + " is a file.");
      }

View on GitHub (pinned to 2add963021)

Solutions

1. If replacement is intended, pass Options.Rename.OVERWRITE: fc.rename(src, dst, Rename.OVERWRITE)
2. If not, pick a unique destination (append timestamp/UUID) or delete dst before renaming
3. For race-prone targets, serialize renames or catch FileAlreadyExistsException and pick a new name instead of retrying blindly

Example fix

// before
fc.rename(src, dst); // dst exists -> FileAlreadyExistsException

// after
fc.rename(src, dst, Options.Rename.OVERWRITE);
// or: fc.delete(dst, true); fc.rename(src, dst);

if (fc.util().exists(dst)) {
  fc.delete(dst, true); // or pass Options.Rename.OVERWRITE, or pick a new name
}

catch (FileAlreadyExistsException e) { /* rename with Rename.OVERWRITE if replacement is safe, else choose a unique name */ }

When it happens

Trigger: fc.rename(src, dst) (Rename.NONE) with dst already present; two tasks racing to rename different sources onto the same target; second execution of an idempotent job writing to a fixed path.

Common situations: Re-running ingestion jobs that stage into a constant destination; concurrent writers in the same directory; leftover output from a killed previous run occupying the target name.

Related errors

• Cannot rename symlink {src} to its target {dst}
• Source {src} and destination {dst} must both be directories
• Rename cannot overwrite non empty destination directory {dst
• Rename destination parent {parent} is a file.
• File system does not support symlinks

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4305d2f7157d9406. Report an issue: GitHub.