{"record":{"id":"f256016497a772f0","repo":"apache/hadoop","slug":"target-dst-already-exists","errorCode":null,"errorMessage":"Target \" + dst + \" already exists","messagePattern":"Target \" \\+ dst \\+ \" already exists","errorType":"exception","errorClass":"PathExistsException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java","lineNumber":625,"sourceCode":"  private static Path checkDest(String srcName, FileSystem dstFS, Path dst,\n      boolean overwrite) throws IOException {\n    FileStatus sdst;\n    try {\n      sdst = dstFS.getFileStatus(dst);\n    } catch (FileNotFoundException e) {\n      sdst = null;\n    }\n    if (null != sdst) {\n      if (sdst.isDirectory()) {\n        if (null == srcName) {\n          if (overwrite) {\n            return dst;\n          }\n          throw new PathIsDirectoryException(dst.toString());\n        }\n        return checkDest(null, dstFS, new Path(dst, srcName), overwrite);\n      } else if (!overwrite) {\n        throw new PathExistsException(dst.toString(),\n            \"Target \" + dst + \" already exists\");\n      }\n    }\n    return dst;\n  }\n\n  public static boolean isRegularFile(File file) {\n    return isRegularFile(file, true);\n  }\n\n  /**\n   * Check if the file is regular.\n   * @param file The file being checked.\n   * @param allowLinks Whether to allow matching links.\n   * @return Returns the result of checking whether the file is a regular file.\n   */\n  public static boolean isRegularFile(File file, boolean allowLinks) {\n    if (file != null) {","sourceCodeStart":607,"sourceCodeEnd":643,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java#L607-L643","documentation":"checkDest() throws PathExistsException(dst.toString(), \"Target <dst> already exists\") when the destination exists as a FILE (not a directory) and overwrite is false. PathExistsException extends IOException. It is the standard 'destination file exists and overwrite is disabled' signal from FileUtil.copy; note the local-to-FileSystem copy() overload hard-codes overwrite=false, so ANY existing target file triggers it.","triggerScenarios":"FileUtil.copy(File src, FileSystem dstFS, Path dst, deleteSource, conf) with an existing file at dst (overwrite is hard-coded false at the call site); re-running a copy job whose output file was left in HDFS.","commonSituations":"Re-executed distcp/put-style jobs without cleanup; idempotent pipelines that assume overwrite but use this API; leftover output from a failed previous run.","solutions":["Delete the existing target first: dstFS.delete(dst, false) when it is a stale file","Switch to an overwrite-aware API: FileSystem.copyFromLocalFile(delSrc, true, src, dst)","Catch PathExistsException explicitly and decide skip/replace/fail per your job semantics","Write to a unique destination (timestamped/UUID name) then atomically publish if you need concurrency"],"exampleFix":"// before\nFileUtil.copy(localSrc, fs, new Path(\"/jobs/out/part-0\"), false, conf);\n// PathExistsException: Target /jobs/out/part-0 already exists\n\n// after\nPath dst = new Path(\"/jobs/out/part-0\");\nif (fs.exists(dst)) {\n  fs.delete(dst, false); // or skip if already correct\n}\nFileUtil.copy(localSrc, fs, dst, false, conf);","handlingStrategy":"try-catch","validationCode":"if (dstFS.exists(dst)) {\n  if (!overwrite) throw new IllegalStateException(\"Target exists: \" + dst);\n  dstFS.delete(dst, false);\n}","typeGuard":null,"tryCatchPattern":"try {\n  FileUtil.copy(src, dstFS, dst, false, conf);\n} catch (PathExistsException e) {\n  // this overload hard-codes overwrite=false; decide skip/replace here\n  dstFS.delete(dst, false);\n  FileUtil.copy(src, dstFS, dst, false, conf);\n}","preventionTips":["Delete or move the prior output file before re-running copy jobs","Use FileSystem.copyFromLocalFile(delSrc, true, src, dst) for overwrite behavior","Write to run-unique filenames and publish atomically for concurrent producers"],"tags":["file-already-exists","copy","overwrite","hdfs","fileutil"],"backgroundTag":"file-already-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}