{"record":{"id":"c6563d084c218e16","repo":"apache/hadoop","slug":"failed-to-rename-s-to-s-file-already-exists-or-c6563d","errorCode":null,"errorMessage":"Failed to rename %s to %s, file already exists or not empty!","messagePattern":"Failed to rename (.+?) to (.+?), file already exists or not empty!","errorType":"exception","errorClass":"FileAlreadyExistsException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java","lineNumber":665,"sourceCode":"        throw new IOException(String.format(\n            \"Failed to rename %s to %s, %s is a file\", srcPath, dstPath,\n            dstPath.getParent()));\n      }\n    } else {\n      if (srcStatus.getPath().equals(dstStatus.getPath())) {\n        return !srcStatus.isDirectory();\n      } else if (dstStatus.isDirectory()) {\n        // If dst is a directory\n        dstPath = new Path(dstPath, srcPath.getName());\n        FileStatus[] statuses;\n        try {\n          statuses = listStatus(dstPath);\n        } catch (FileNotFoundException fnde) {\n          statuses = null;\n        }\n        if (statuses != null && statuses.length > 0) {\n          // If dst exists and not a directory / not empty\n          throw new FileAlreadyExistsException(String.format(\n              \"Failed to rename %s to %s, file already exists or not empty!\",\n              srcPath, dstPath));\n        }\n      } else {\n        // If dst is not a directory\n        throw new FileAlreadyExistsException(String.format(\n            \"Failed to rename %s to %s, file already exists!\", srcPath,\n            dstPath));\n      }\n    }\n\n    boolean succeed;\n    if (srcStatus.isDirectory()) {\n      succeed = copyDirectory(srcPath, dstPath);\n    } else {\n      succeed = copyFile(srcPath, srcStatus.getLen(), dstPath);\n    }\n","sourceCodeStart":647,"sourceCodeEnd":683,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java#L647-L683","documentation":"Thrown by AliyunOSSFileSystem.rename(): the destination exists and is a directory, so the source is renamed into it as dst/srcName — but listStatus on that joined path found existing entries. To honor POSIX rename-on-directory semantics (destination directory must be empty), it throws FileAlreadyExistsException(\"Failed to rename %s to %s, file already exists or not empty!\").","triggerScenarios":"Calling rename(src, dstDir) where an object already exists at dstDir/srcName; e.g., rename(oss://b/src, oss://b/dst) when object 'dst/src' already exists — common in Hive partition moves and committer staging where the target was partially populated.","commonSituations":"Rerunning a job after a partially failed commit that left the target in place; concurrent writers moving files into the same directory; incremental pipelines that assume overwrite-on-rename (OSS rename is copy+delete and does not overwrite).","solutions":["Delete the existing entry at dst/srcName before renaming, or choose a fresh destination name","Make rename targets idempotent-unique (attempt id / timestamp suffix) so reruns cannot collide","If the leftover came from a failed commit, clean the destination subtree before the retry"],"exampleFix":"// before\nfs.rename(new Path(\"oss://b/src\"), new Path(\"oss://b/dst\")); // dst/src exists\n\n// after\nPath target = new Path(\"oss://b/dst/src\");\nif (fs.exists(target)) {\n  fs.delete(target, false);\n}\nfs.rename(new Path(\"oss://b/src\"), new Path(\"oss://b/dst\"));","handlingStrategy":"validation","validationCode":"Path target = dst;\nif (fs.exists(dst) && fs.getFileStatus(dst).isDirectory()) {\n  target = new Path(dst, src.getName());\n}\nif (fs.exists(target)) {\n  throw new IOException(\"Rename target already populated: \" + target);\n}\nfs.rename(src, dst);","typeGuard":null,"tryCatchPattern":"catch (FileAlreadyExistsException e) { /* dst/srcName exists: delete it or pick a fresh name, then retry once */ throw e; }","preventionTips":["Pre-compute the effective destination (dst/srcName when dst is a dir) and check emptiness before rename","Make commit/move stages idempotent with unique target names","Clean partially-committed destinations before reruns"],"tags":["aliyun-oss","rename","destination-not-empty","hadoop-connector"],"backgroundTag":"rename-target-not-empty","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}