{"record":{"id":"3de9daa31059ab6c","repo":"apache/hadoop","slug":"destination-is-a-non-empty-directory","errorCode":null,"errorMessage":"Destination is a non-empty directory","messagePattern":"Destination is a non-empty directory","errorType":"exception","errorClass":"RenameFailedException","httpStatus":null,"severity":"warning","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java","lineNumber":2467,"sourceCode":"      throw new RenameFailedException(src, dst,\n          \"source and dest refer to the same file or directory\")\n          .withExitCode(srcStatus.isFile());\n    }\n\n    S3AFileStatus dstStatus = null;\n    try {\n      dstStatus = innerGetFileStatus(dst, true, StatusProbeEnum.ALL);\n      // if there is no destination entry, an exception is raised.\n      // hence this code sequence can assume that there is something\n      // at the end of the path; the only detail being what it is and\n      // whether or not it can be the destination of the rename.\n      if (srcStatus.isDirectory()) {\n        if (dstStatus.isFile()) {\n          throw new FileAlreadyExistsException(\n              \"Failed to rename \" + src + \" to \" + dst\n               +\"; source is a directory and dest is a file\");\n        } else if (dstStatus.isEmptyDirectory() != Tristate.TRUE) {\n          throw new RenameFailedException(src, dst,\n              \"Destination is a non-empty directory\")\n              .withExitCode(false);\n        }\n        // at this point the destination is an empty directory\n      } else {\n        // source is a file. The destination must be a directory,\n        // empty or not\n        if (dstStatus.isFile()) {\n          throw new FileAlreadyExistsException(\n              \"Failed to rename \" + src + \" to \" + dst\n                  + \"; destination file exists\");\n        }\n      }\n\n    } catch (FileNotFoundException e) {\n      LOG.debug(\"rename: destination path {} not found\", dst);\n      // Parent must exist\n      Path parent = dst.getParent();","sourceCodeStart":2449,"sourceCodeEnd":2485,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2449-L2485","documentation":"Source is a directory and the destination exists as a directory that is not provably empty (dstStatus.isEmptyDirectory() != Tristate.TRUE covers both non-empty and 'unknown', since marker-less directories cannot be proven empty). RenameFailedException 'Destination is a non-empty directory' is thrown with exit code false; rename() catches it, logs INFO, and returns false - matching POSIX rename(2) ENOTEMPTY/EEXIST semantics.","triggerScenarios":"rename(srcDir, dstDir) where dstDir contains objects or a KEEP marker, or its emptiness cannot be determined from the listing; large trees where directory markers are not authoritative.","commonSituations":"Attempting directory 'merge' via rename (not supported); destination not cleaned from a previous run; directory marker retention policies making emptiness indeterminate.","solutions":["Clean the destination first: fs.delete(dst, true), then rename","Use a fresh destination path (timestamped) instead of merging","For merge semantics, copy children individually (FileUtil.copy, distcp) rather than rename"],"exampleFix":"// before: returns false when dst is non-empty\nboolean ok = fs.rename(src, dst);\n\n// after: explicit cleanup makes intent visible\nif (fs.exists(dst)) {\n  fs.delete(dst, true);\n}\nboolean ok = fs.rename(src, dst);","handlingStrategy":"validation","validationCode":"static boolean destinationAcceptsRename(FileSystem fs, Path dst) throws IOException {\n  if (!fs.exists(dst)) return true;\n  FileStatus st = fs.getFileStatus(dst);\n  return st.isDirectory() && st instanceof S3AFileStatus\n      && ((S3AFileStatus) st).isEmptyDirectory() == Tristate.TRUE;\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Delete or version destination directories before renaming into them","Do not use rename to merge directories - copy children instead","Treat rename()==false with destination 'unknown emptiness' as a hard precondition, and list the destination to decide"],"tags":["s3a","rename","directory-not-empty","posix-semantics"],"backgroundTag":"directory-not-empty","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}