{"record":{"id":"14f1cd99687949ec","repo":"apache/hadoop","slug":"failed-to-delete","errorCode":null,"errorMessage":"Failed to delete {}","messagePattern":"Failed to delete (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java","lineNumber":477,"sourceCode":"   * @throws IOException on any error\n   */\n  private void mergePaths(FileSystem fs, final FileStatus from,\n      final Path to, JobContext context) throws IOException {\n    try (DurationInfo d = new DurationInfo(LOG,\n        false,\n        \"Merging data from %s to %s\", from, to)) {\n      reportProgress(context);\n      FileStatus toStat;\n      try {\n        toStat = fs.getFileStatus(to);\n      } catch (FileNotFoundException fnfe) {\n        toStat = null;\n      }\n\n      if (from.isFile()) {\n        if (toStat != null) {\n          if (!fs.delete(to, true)) {\n            throw new IOException(\"Failed to delete \" + to);\n          }\n        }\n\n        if (!fs.rename(from.getPath(), to)) {\n          throw new IOException(\"Failed to rename \" + from + \" to \" + to);\n        }\n      } else if (from.isDirectory()) {\n        if (toStat != null) {\n          if (!toStat.isDirectory()) {\n            if (!fs.delete(to, true)) {\n              throw new IOException(\"Failed to delete \" + to);\n            }\n            renameOrMerge(fs, from, to, context);\n          } else {\n            //It is a directory so merge everything in the directories\n            for (FileStatus subFrom : fs.listStatus(from.getPath())) {\n              Path subTo = new Path(to, subFrom.getPath().getName());\n              mergePaths(fs, subFrom, subTo, context);","sourceCodeStart":459,"sourceCodeEnd":495,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java#L459-L495","documentation":"During commit, FileOutputCommitter.mergePaths (FileOutputCommitter.java:461+) moves task-attempt output into the final output directory. When the destination path already exists, it first deletes it with fs.delete(to, true); if the FileSystem returns false (delete failed without throwing), the committer throws IOException('Failed to delete <to>'). This is a commit-phase failure, typically surfaced in task/AM commit or job-commit logs.","triggerScenarios":"Destination file/dir already exists in the output directory and cannot be deleted: permissions lacking on the parent, read-only or safe-mode HDFS, another job concurrently writing the same output path, a stale _temporary directory from a previous failed attempt containing the same names, or object-store semantics (S3A) where delete/rename behave differently.","commonSituations":"Rerunning jobs into an existing output directory that contains leftovers; two concurrent jobs sharing one output path; speculative tasks or AM retry racing on commit; misconfigured permissions on the output dir; using algorithm version 1 vs 2 with shared output locations; S3 without a proper S3A committer configured.","solutions":["Clean the output directory before rerunning: hadoop fs -rm -r /output (or write to a new output path per run)","Verify write+delete permissions on the output directory for the running user (hadoop fs -ls, -touchz a scratch file then delete it)","Prevent concurrent jobs from sharing an output path; add unique run IDs/staging dirs to output locations","Check fs health at failure time: HDFS safe mode (hdfs dfsadmin -safemode get), NameNode failover mid-commit; retry the job after the cluster is healthy","For S3 outputs, configure a real S3A committer (magic or directory committer via fs.s3a.committer.name) instead of relying on FileOutputCommitter renames"],"exampleFix":"# before\nhadoop jar app.jar Driver /in /out   # /out still contains committed files from previous failed run\n\n# after\nhadoop fs -rm -r /out/_temporary 2>/dev/null\nhadoop fs -rm -r /out               # or: /out-$(date +%s) per-run output\nhadoop jar app.jar Driver /in /out","handlingStrategy":"retry","validationCode":"static void requireCleanWritableOutput(Path out, Configuration conf) throws IOException {\n  FileSystem fs = out.getFileSystem(conf);\n  if (fs.exists(out)) {\n    FileStatus st = fs.getFileStatus(out);\n    if (!st.getPermission().getUserAction().implies(FsAction.WRITE))\n      throw new IllegalStateException(\"Output dir not writable: \" + out);\n    if (fs.exists(new Path(out, \"_temporary\")))\n      throw new IllegalStateException(\"Stale _temporary present; remove it or use a fresh output dir\");\n  }\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) { if (String.valueOf(e.getMessage()).startsWith(\"Failed to delete\")) { // transient FS states (safe mode, failover) often clear: clean output, verify permissions, retry job once\n  throw new IOException(\"Commit-time delete failed for \" + e.getMessage() + \" — clean output dir, check permissions/safe-mode, rerun\", e); } throw e; }","preventionTips":["Always write to a fresh (or explicitly cleaned) output directory per run","Ensure the submitting user can delete under the output path (check perms + owner)","Never point concurrent jobs at the same output directory","For object stores use store-native committers instead of rename/delete-based commit"],"tags":["hadoop","mapreduce","output-committer","commit-phase","filesystem"],"backgroundTag":"commit-delete-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}