apache/hadoop · error · PathExistsException

Target " + dst + " already exists

Error message

Target " + dst + " already exists

What it means

checkDest() throws PathExistsException(dst.toString(), "Target <dst> already exists") when the destination exists as a FILE (not a directory) and overwrite is false. PathExistsException extends IOException. It is the standard 'destination file exists and overwrite is disabled' signal from FileUtil.copy; note the local-to-FileSystem copy() overload hard-codes overwrite=false, so ANY existing target file triggers it.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java:625

  private static Path checkDest(String srcName, FileSystem dstFS, Path dst,
      boolean overwrite) throws IOException {
    FileStatus sdst;
    try {
      sdst = dstFS.getFileStatus(dst);
    } catch (FileNotFoundException e) {
      sdst = null;
    }
    if (null != sdst) {
      if (sdst.isDirectory()) {
        if (null == srcName) {
          if (overwrite) {
            return dst;
          }
          throw new PathIsDirectoryException(dst.toString());
        }
        return checkDest(null, dstFS, new Path(dst, srcName), overwrite);
      } else if (!overwrite) {
        throw new PathExistsException(dst.toString(),
            "Target " + dst + " already exists");
      }
    }
    return dst;
  }

  public static boolean isRegularFile(File file) {
    return isRegularFile(file, true);
  }

  /**
   * Check if the file is regular.
   * @param file The file being checked.
   * @param allowLinks Whether to allow matching links.
   * @return Returns the result of checking whether the file is a regular file.
   */
  public static boolean isRegularFile(File file, boolean allowLinks) {
    if (file != null) {

View on GitHub (pinned to 2add963021)

Solutions

1. Delete the existing target first: dstFS.delete(dst, false) when it is a stale file
2. Switch to an overwrite-aware API: FileSystem.copyFromLocalFile(delSrc, true, src, dst)
3. Catch PathExistsException explicitly and decide skip/replace/fail per your job semantics
4. Write to a unique destination (timestamped/UUID name) then atomically publish if you need concurrency

Example fix

// before
FileUtil.copy(localSrc, fs, new Path("/jobs/out/part-0"), false, conf);
// PathExistsException: Target /jobs/out/part-0 already exists

// after
Path dst = new Path("/jobs/out/part-0");
if (fs.exists(dst)) {
  fs.delete(dst, false); // or skip if already correct
}
FileUtil.copy(localSrc, fs, dst, false, conf);

if (dstFS.exists(dst)) {
  if (!overwrite) throw new IllegalStateException("Target exists: " + dst);
  dstFS.delete(dst, false);
}

try {
  FileUtil.copy(src, dstFS, dst, false, conf);
} catch (PathExistsException e) {
  // this overload hard-codes overwrite=false; decide skip/replace here
  dstFS.delete(dst, false);
  FileUtil.copy(src, dstFS, dst, false, conf);
}

When it happens

Trigger: FileUtil.copy(File src, FileSystem dstFS, Path dst, deleteSource, conf) with an existing file at dst (overwrite is hard-coded false at the call site); re-running a copy job whose output file was left in HDFS.

Common situations: Re-executed distcp/put-style jobs without cleanup; idempotent pipelines that assume overwrite but use this API; leftover output from a failed previous run.

Related errors

• dst.toString()
• File exists
• Object %s already exists.
• {} already exists
• src.toString() + ": No such file or directory"

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f256016497a772f0. Report an issue: GitHub.