{"record":{"id":"9920f1269c01126e","repo":"apache/hadoop","slug":"rename-from-to-failed-9920f1","errorCode":null,"errorMessage":"rename from {} to {} failed.","messagePattern":"rename from (.+?) to (.+?) failed\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java","lineNumber":497,"sourceCode":"          // deleted. Need to update the SnapshotManager.\n          fsd.getFSNamesystem().removeSnapshottableDirs(dstSnapshottableDirs);\n        }\n\n        tx.updateQuotasInSourceTree(bsps);\n        return createRenameResult(\n            fsd, renamedIIP, filesDeleted, collectedBlocks);\n      }\n    } finally {\n      if (undoRemoveSrc) {\n        tx.restoreSource();\n      }\n      if (undoRemoveDst) { // Rename failed - restore dst\n        tx.restoreDst(bsps);\n      }\n    }\n    NameNode.stateChangeLog.warn(\"DIR* FSDirectory.unprotectedRenameTo: \" +\n        \"failed to rename \" + src + \" to \" + dst);\n    throw new IOException(\"rename from \" + src + \" to \" + dst + \" failed.\");\n  }\n\n  /**\n   * @deprecated Use {@link #renameToInt(FSDirectory, FSPermissionChecker,\n   * String, String, boolean, Options.Rename...)}\n   */\n  @Deprecated\n  private static RenameResult renameTo(FSDirectory fsd, FSPermissionChecker pc,\n      INodesInPath srcIIP, INodesInPath dstIIP, boolean logRetryCache)\n          throws IOException {\n    if(fsd.isNonEmptyDirectory(srcIIP)) {\n      DFSUtil.checkProtectedDescendants(fsd, srcIIP);\n    }\n\n    if (fsd.isPermissionEnabled()) {\n      // Check write access to parent of src\n      fsd.checkPermission(pc, srcIIP, false, null, FsAction.WRITE, null, null,\n          false);","sourceCodeStart":479,"sourceCodeEnd":515,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java#L479-L515","documentation":"IOException thrown at the end of a failed two-phase rename after the NameNode already rolled back (tx.restoreSource/restoreDst in the finally block). It means the source was detached but re-attaching it at the destination failed - classically another client created dst concurrently in the non-overwrite path - so the namespace was restored to the pre-rename state and the failure reported. State stays consistent; the operation just lost a race.","triggerScenarios":"Two clients renaming to the same destination concurrently; commit protocols where multiple tasks race the final rename to a common path; overwrite=false while the destination appeared between validation and addLastINode.","commonSituations":"Atomic-output commit races (two tasks both attempt the final move); distcp or balancer-like tooling contending with live jobs; aggressive client retries that recreate the destination.","solutions":["Retry the rename after re-resolving; the race has usually cleared.","If the destination legitimately exists, use the overwrite variant: fs.rename(src, dst, Options.Rename.OVERWRITE).","Design commit paths so only one writer performs the final rename (unique temp names plus a single renamer)."],"exampleFix":"// before\nfs.rename(src, dst); // IOException: rename from X to Y failed (lost race)\n\n// after\nfor (int i = 0; i < 3; i++) {\n  try { fs.rename(src, dst); break; }\n  catch (IOException e) {\n    if (!fs.exists(src) || i == 2) throw e;\n    if (fs.exists(dst)) { fs.rename(src, dst, Options.Rename.OVERWRITE); break; }\n  }\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"static void renameWithRetry(FileSystem fs, Path src, Path dst, int attempts) throws IOException {\n  for (int i = 0; i < attempts; i++) {\n    try {\n      fs.rename(src, dst);\n      return;\n    } catch (IOException e) {\n      if (!fs.exists(src)) throw e;          // source really gone: not a race\n      if (fs.exists(dst)) {                  // destination appeared concurrently\n        fs.rename(src, dst, Options.Rename.OVERWRITE);\n        return;\n      }\n      if (i == attempts - 1) throw e;\n    }\n  }\n}","preventionTips":["Use unique temporary names so only one task performs the final rename (classic commit protocol).","Prefer the Options.Rename.OVERWRITE form when the destination may legitimately exist.","Treat this IOException as retryable only while src still exists; otherwise surface it."],"tags":["hdfs","rename","race-condition","transaction-rollback"],"backgroundTag":"rename-transaction-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}