{"record":{"id":"756b1169e7368e01","repo":"apache/hadoop","slug":"failed-to-rename-to","errorCode":null,"errorMessage":"Failed to rename {} to {}","messagePattern":"Failed to rename (.+?) to (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java","lineNumber":582,"sourceCode":"      }\n\n      if (restoreDirectory == null) {\n        restoreDirectory = new File(getRestoreDirectory(child));\n        if (!restoreDirectory.exists() && !restoreDirectory.mkdirs()) {\n          throw new IOException(\"Failed to create directory \" + restoreDirectory);\n        }\n      }\n\n      final File newChild = new File(restoreDirectory, child.getName());\n\n      if (newChild.exists() && newChild.length() >= child.length()) {\n        // Failsafe - we should not hit this case but let's make sure\n        // we never overwrite a newer version of a block file with an\n        // older version.\n        LOG.info(\"Not overwriting {} with smaller file from \" +\n            \"trash directory. This message can be safely ignored.\", newChild);\n      } else if (!child.renameTo(newChild)) {\n        throw new IOException(\"Failed to rename \" + child + \" to \" + newChild);\n      } else {\n        ++filesRestored;\n      }\n    }\n    FileUtil.fullyDelete(trashRoot);\n    return filesRestored;\n  }\n\n  /*\n   * Roll back to old snapshot at the block pool level\n   * If previous directory exists: \n   * <ol>\n   * <li>Rename <SD>/current/<bpid>/current to removed.tmp</li>\n   * <li>Rename * <SD>/current/<bpid>/previous to current</li>\n   * <li>Remove removed.tmp</li>\n   * </ol>\n   * \n   * Do nothing if previous directory does not exist.","sourceCodeStart":564,"sourceCodeEnd":600,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java#L564-L600","documentation":"In the same trash-restore loop, renaming a trashed block file back to its restore location failed (File.renameTo returned false) — the block file could not be moved back into current/, so the rollback aborts.","triggerScenarios":"Permission denied on source or target; another process holds the file open; target anomaly or filesystem (NFS/FUSE) refusing the rename; out-of-space on metadata updates.","commonSituations":"Rolling-upgrade rollback with AV/scanners or backup agents touching block files; DN user mismatch; NAS-backed storage.","solutions":["Verify the DN user owns both the trash entry and the target dir and nothing holds the file open (lsof)","Copy the file manually (cp, then confirm identical length) and delete the trash copy if rename keeps failing","Fix the underlying FS issue (space, mount flags) and retry the rollback"],"exampleFix":"# cp is a safe manual fallback to rename during trash restore:\ncp -p /dfs/dn/current/BP-*/trash/subdir/blk_x /dfs/dn/current/BP-*/current/finalized/subdir/blk_x\ncmp blk_x blk_x   # lengths/bytes equal\nhdfs --daemon start datanode -rollback","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (e.getMessage().contains(\"Failed to rename\")) {\n    // fall back to copy+verify+delete, which achieves the same restore\n    restoreByCopy(child, newChild);\n  }\n}","preventionTips":["Exclude DN block files from AV/backup scans that can hold file handles during rollback","Use local filesystems (not NFS/FUSE) for dfs.datanode.data.dir","Verify the DN user owns both the trash and finalized trees before rollback"],"tags":["hdfs","datanode","storage","rollback","filesystem"],"backgroundTag":"filesystem-permission-denied","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}