{"record":{"id":"9e8f46fc104b1a07","repo":"apache/hadoop","slug":"failed-to-rename-src-to-dst-destination-file","errorCode":null,"errorMessage":"Failed to rename <src> to <dst>; destination file exists","messagePattern":"Failed to rename <src> to <dst>; destination file exists","errorType":"exception","errorClass":"FileAlreadyExistsException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java","lineNumber":2476,"sourceCode":"      // hence this code sequence can assume that there is something\n      // at the end of the path; the only detail being what it is and\n      // whether or not it can be the destination of the rename.\n      if (srcStatus.isDirectory()) {\n        if (dstStatus.isFile()) {\n          throw new FileAlreadyExistsException(\n              \"Failed to rename \" + src + \" to \" + dst\n               +\"; source is a directory and dest is a file\");\n        } else if (dstStatus.isEmptyDirectory() != Tristate.TRUE) {\n          throw new RenameFailedException(src, dst,\n              \"Destination is a non-empty directory\")\n              .withExitCode(false);\n        }\n        // at this point the destination is an empty directory\n      } else {\n        // source is a file. The destination must be a directory,\n        // empty or not\n        if (dstStatus.isFile()) {\n          throw new FileAlreadyExistsException(\n              \"Failed to rename \" + src + \" to \" + dst\n                  + \"; destination file exists\");\n        }\n      }\n\n    } catch (FileNotFoundException e) {\n      LOG.debug(\"rename: destination path {} not found\", dst);\n      // Parent must exist\n      Path parent = dst.getParent();\n      if (!pathToKey(parent).isEmpty()\n          && !parent.equals(src.getParent())) {\n        try {\n          // make sure parent isn't a file.\n          // don't look for parent being a dir as there is a risk\n          // of a race between dest dir cleanup and rename in different\n          // threads.\n          S3AFileStatus dstParentStatus = innerGetFileStatus(parent,\n              false, StatusProbeEnum.FILE);","sourceCodeStart":2458,"sourceCodeEnd":2494,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2458-L2494","documentation":"In initiateRename(), the source is a file and the destination also exists as a file. Hadoop's rename contract never overwrites an existing destination file, so FileAlreadyExistsException 'Failed to rename <src> to <dst>; destination file exists' is thrown and propagates out of rename() (it is not converted to a false return like RenameFailedException).","triggerScenarios":"rename(file, file) where an object already exists at dst; two writers renaming different sources onto the same published name.","commonSituations":"Output already committed by a previous run; concurrent tasks racing to publish the same final path; publishing logic without unique staging names.","solutions":["Delete the destination object (or the whole output tree in job cleanup) before renaming","Publish via unique staging names plus an atomic final rename, or conditional create","Pre-check fs.exists(dst) and decide explicitly whether to replace"],"exampleFix":"// before: throws if dst object exists\nfs.rename(staged, finalPath);\n\n// after: remove stale destination deliberately\nif (fs.exists(finalPath)) {\n  fs.delete(finalPath, false);\n}\nfs.rename(staged, finalPath);","handlingStrategy":"validation","validationCode":"if (fs.exists(dst)) {\n  fs.delete(dst, false);\n}\nboolean ok = fs.rename(src, dst);","typeGuard":null,"tryCatchPattern":"Catch FileAlreadyExistsException from rename() in publish paths and treat it as a duplicate-publish signal: either the previous run's output is stale (delete it deliberately) or a concurrent publisher won the race (skip).","preventionTips":["Publish via unique staging names and one final rename","Clean committed output in job setup","Pre-check fs.exists(dst) where overwrite is intended"],"tags":["s3a","rename","file-already-exists","publish"],"backgroundTag":"file-already-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}