{"record":{"id":"479a752a7b3d8b14","repo":"apache/hadoop","slug":"fail-to-rename-tmp-file-tmp-to-destination-fi","errorCode":null,"errorMessage":"Fail to rename tmp file (={tmp}) to destination file (={dst})","messagePattern":"Fail to rename tmp file \\(=(.+?)\\) to destination file \\(=(.+?)\\)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java","lineNumber":674,"sourceCode":"            targetFile, skipCrc, srcFileStatus.getLen());\n  }\n\n  /**\n   * Rename tmp to dst on destFileSys.\n   * @param destFileSys the file ssystem\n   * @param tmp the source path\n   * @param dst the destination path\n   * @throws IOException if renaming failed\n   */\n  private static void rename(FileSystem destFileSys, Path tmp, Path dst)\n      throws IOException {\n    try {\n      if (destFileSys.exists(dst)) {\n        destFileSys.delete(dst, true);\n      }\n      destFileSys.rename(tmp, dst);\n    } catch (IOException ioe) {\n      throw new IOException(\"Fail to rename tmp file (=\" + tmp\n          + \") to destination file (=\" + dst + \")\", ioe);\n    }\n  }\n\n}\n","sourceCodeStart":656,"sourceCodeEnd":680,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java#L656-L680","documentation":"CopyCommitter.concatFileChunks() (used with the -blocksperchunk chunked-copy mode) concatenates copied chunks and renames the result onto the target file through this rename() helper. If exists(), delete() or rename() on the destination throws, the IOException is re-wrapped with the tmp and destination paths. The real failure is in the nested cause.","triggerScenarios":"Another writer holds or recreates the destination between the delete and the rename; permission lost on the destination parent; the tmp chunk file removed by an external cleaner; two concurrent distcp jobs committing the same target file.","commonSituations":"Overlapping distcp runs with -blocksperchunk aimed at the same destination; cron cleanup deleting .distcp.tmp.* files mid-commit; ACL or ownership changes on the target directory during a long job.","solutions":["Walk the nested cause in the job log to see whether delete or rename failed and why","Ensure only one job writes a given target file at a time; serialize distcp runs that share a destination","Stop or scope cleanup jobs so they cannot delete .distcp.tmp.* or chunk files while distcp runs","Re-run with -update so completed files are skipped and only the failed target is re-copied"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch (IOException e) {  // wraps 'Fail to rename tmp file'\n  Throwable root = e;\n  while (root.getCause() != null) root = root.getCause();\n  LOG.error(\"chunk concat/rename failed, root cause:\", root);\n}","preventionTips":["Serialize distcp runs that share a destination file","Keep cleanup jobs away from .distcp.tmp.* files during jobs","Verify ACLs/ownership on the target directory before large chunked copies","Use -update on re-runs so finished files are skipped"],"tags":["distcp","hadoop","rename","concat","blocksperchunk"],"backgroundTag":"rename-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}