apache/hadoop · error · FileAlreadyExistsException

File: %s already exists

Error message

File: %s already exists

What it means

When a rename destination exists and is a FILE, CosN throws FileAlreadyExistsException('File: <dst> already exists') rather than silently overwriting. Rename in CosN is implemented as copy+delete, which cannot atomically replace an object, so destination collisions are rejected outright. Note the classic Hadoop contract permits returning false here; CosN chooses to fail fast instead.

Source

Thrown at hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNFileSystem.java:651

          dstParentPath);
    }

    if (null != dstParentPath) {
      LOG.debug("It is not allowed to rename a parent directory:[{}] "
          + "to its subdirectory:[{}].", src, dst);
      throw new IOException(String.format(
          "It is not allowed to rename a parent directory: %s "
              + "to its subdirectory: %s", src, dst));
    }

    FileStatus dstFileStatus;
    try {
      dstFileStatus = this.getFileStatus(dst);

      // The destination path exists and is a file,
      // and the rename operation is not allowed.
      if (dstFileStatus.isFile()) {
        throw new FileAlreadyExistsException(String.format(
            "File: %s already exists", dstFileStatus.getPath()));
      } else {
        // The destination path is an existing directory,
        // and it is checked whether there is a file or directory
        // with the same name as the source path under the destination path
        dst = new Path(dst, src.getName());
        FileStatus[] statuses;
        try {
          statuses = this.listStatus(dst);
        } catch (FileNotFoundException e) {
          statuses = null;
        }
        if (null != statuses && statuses.length > 0) {
          LOG.debug("Cannot rename source file: [{}] to dest file: [{}], "
              + "because the file already exists.", src, dst);
          throw new FileAlreadyExistsException(
              String.format(
                  "File: %s already exists", dst

View on GitHub (pinned to 2add963021)

Solutions

1. If overwrite is intended: fs.delete(dst, false) first, then rename (accepting the small non-atomic window).
2. Write to temporary names and only rename to a final name you have verified is absent.
3. Make the step idempotent: treat 'dst already exists with matching content/length' as success and skip.

Example fix

// before
fs.rename(tmpPath, finalPath); // FileAlreadyExistsException: File: ... already exists

// after
if (fs.exists(finalPath)) {
  if (fs.getFileStatus(finalPath).getLen() != fs.getFileStatus(tmpPath).getLen()) {
    fs.delete(finalPath, false);
    fs.rename(tmpPath, finalPath);
  } // else: already committed, skip
} else {
  fs.rename(tmpPath, finalPath);
}

if (fs.exists(dst) && fs.getFileStatus(dst).isFile()) {
  if (!allowOverwrite) { throw new IllegalStateException('Destination exists: ' + dst); }
  fs.delete(dst, false);
}
fs.rename(src, dst);

static boolean isRenameCollision(Throwable t) {
  return t instanceof FileAlreadyExistsException;
}

try {
  fs.rename(src, dst);
} catch (FileAlreadyExistsException e) {
  // idempotent retry semantics: same length means already committed
  if (fs.getFileStatus(dst).getLen() == expectedLen) { /* done */ } else { throw e; }
}

When it happens

Trigger: fs.rename(src, dst) where dst resolves to an existing file; hadoop fs -mv onto an existing file name; idempotent retry of a rename step after a partial failure where dst was already committed but src was recreated.

Common situations: Job-step retries after partial failure; two writers racing to promote the same final name; habits carried over from local fs -mv where the destination is replaced.

Related errors

• {} already exists
• Not a directory: {}
• Path is a file: {}
• Can't make directory for path '%s' since it is a file.
• Source path and dest path refer the same file or directory

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/74f14fa9f2d49c82. Report an issue: GitHub.