{"record":{"id":"bbde0ad8121896b8","repo":"apache/hadoop","slug":"rename-cannot-overwrite-non-empty-destination-dire","errorCode":null,"errorMessage":"Rename cannot overwrite non empty destination directory {dst}","messagePattern":"Rename cannot overwrite non empty destination directory (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java","lineNumber":883,"sourceCode":"      }\n      if (srcStatus.isSymlink() && dst.equals(srcStatus.getSymlink())) {\n        throw new FileAlreadyExistsException(\n            \"Cannot rename symlink \"+src+\" to its target \"+dst);\n      }\n      // It's OK to rename a file to a symlink and vice versa\n      if (srcStatus.isDirectory() != dstStatus.isDirectory()) {\n        throw new IOException(\"Source \" + src + \" and destination \" + dst\n            + \" must both be directories\");\n      }\n      if (!overwrite) {\n        throw new FileAlreadyExistsException(\"Rename destination \" + dst\n            + \" already exists.\");\n      }\n      // Delete the destination that is a file or an empty directory\n      if (dstStatus.isDirectory()) {\n        RemoteIterator<FileStatus> list = listStatusIterator(dst);\n        if (list != null && list.hasNext()) {\n          throw new IOException(\n              \"Rename cannot overwrite non empty destination directory \" + dst);\n        }\n      }\n      delete(dst, false);\n    } else {\n      final Path parent = dst.getParent();\n      final FileStatus parentStatus = getFileStatus(parent);\n      if (parentStatus.isFile()) {\n        throw new ParentNotDirectoryException(\"Rename destination parent \"\n            + parent + \" is a file.\");\n      }\n    }\n    renameInternal(src, dst);\n  }\n  \n  /**\n   * Returns true if the file system supports symlinks, false otherwise.\n   * @return true if filesystem supports symlinks","sourceCodeStart":865,"sourceCodeEnd":901,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java#L865-L901","documentation":"With Rename.OVERWRITE, Hadoop will only replace a destination directory that is empty: rename is an atomic replace, not a merge. renameInternal probes dst with listStatusIterator and throws this IOException as soon as one entry is returned, mirroring the HDFS NameNode's own non-empty-directory check.","triggerScenarios":"fc.rename(srcDir, dstDir, Rename.OVERWRITE) where dstDir contains files or subdirectories; sync tools that assume rsync-style merge semantics; directories that look empty but hold hidden entries (.crc, _SUCCESS, .tmp part files).","commonSituations":"Atomic output committers publishing into a partition directory that still contains previous files; recovery after a partially committed earlier run; republishing dated partition paths.","solutions":["Delete the destination tree first: fc.delete(dst, true), then perform the overwrite rename","Merge contents yourself (copy/move old entries aside) before overwriting","Use versioned destination names plus a pointer/alias switch instead of in-place overwrite"],"exampleFix":"// before\nfc.rename(srcDir, dstDir, Options.Rename.OVERWRITE); // dstDir non-empty -> IOException\n\n// after\nif (fc.util().exists(dstDir)) {\n  fc.delete(dstDir, true); // only if discarding old contents is intended\n}\nfc.rename(srcDir, dstDir, Options.Rename.OVERWRITE);","handlingStrategy":"validation","validationCode":"if (fc.util().exists(dst)) {\n  FileStatus[] entries = fc.util().listStatus(dst);\n  if (entries.length > 0) {\n    fc.delete(dst, true); // only when discarding old contents is intended\n  }\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) { if (e.getMessage().contains(\"non empty destination\")) { /* delete dst recursively, then retry once */ } else throw e; }","preventionTips":["Treat overwrite-rename as replace, not merge: verify the destination directory is empty first","Remember hidden entries (.crc, _SUCCESS, .tmp) count as non-empty","Prefer versioned output directories plus pointer swap over in-place overwrites"],"tags":["rename","overwrite","directory-not-empty","filecontext","hadoop-fs"],"backgroundTag":"directory-not-empty","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}