{"record":{"id":"d9184fe88c011a67","repo":"apache/hadoop","slug":"new-destination-is-an-existed-file","errorCode":null,"errorMessage":"new destination is an existed file","messagePattern":"new destination is an existed file","errorType":"exception","errorClass":"RenameFailedException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSObjectBucketUtils.java","lineNumber":131,"sourceCode":"      // if there is no destination entry, an exception is raised.\n      // hence this code sequence can assume that there is something\n      // at the end of the path; the only detail being what it is and\n      // whether or not it can be the destination of the rename.\n      if (dstStatus.isDirectory()) {\n        String newDstKey = OBSCommonUtils.maybeAddTrailingSlash(dstKey);\n        String filename = srcKey.substring(\n            OBSCommonUtils.pathToKey(owner, src.getParent()).length()\n                + 1);\n        newDstKey = newDstKey + filename;\n        dstKey = newDstKey;\n        dstStatus = owner.getFileStatus(\n            OBSCommonUtils.keyToPath(dstKey));\n        if (dstStatus.isDirectory()) {\n          throw new RenameFailedException(src, dst,\n              \"new destination is an existed directory\")\n              .withExitCode(false);\n        } else {\n          throw new RenameFailedException(src, dst,\n              \"new destination is an existed file\")\n              .withExitCode(false);\n        }\n      } else {\n\n        if (srcKey.equals(dstKey)) {\n          LOG.warn(\n              \"rename: src and dest refer to the same file or\"\n                  + \" directory: {}\",\n              dst);\n          return true;\n        } else {\n          throw new RenameFailedException(src, dst,\n              \"destination is an existed file\")\n              .withExitCode(false);\n        }\n      }\n    } catch (FileNotFoundException e) {","sourceCodeStart":113,"sourceCodeEnd":149,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSObjectBucketUtils.java#L113-L149","documentation":"When the destination of a rename exists and is a directory, the effective target becomes dst/<srcname>. renameBasedObject stats that composed key and, if it is an existing file, throws RenameFailedException('new destination is an existed file').withExitCode(false). Object storage cannot atomically overwrite a file during a directory-style rename, so the operation is refused.","triggerScenarios":"fs.rename('/src/file', '/dst') when /dst/file already exists as a file; two clients moving different files named identically into the same directory; commit-by-rename re-executed after the destination file was already written.","commonSituations":"Write-once-then-commit pipelines on object storage where POSIX overwrite semantics were assumed; concurrent Spark task attempts committing to the same output subdirectory; retry after a network failure where the first attempt actually succeeded.","solutions":["Check for dst/<srcname> with getFileStatus before renaming and delete it when overwrite is intended","Treat 'target file exists and source still exists' as a signal of a prior partial run: verify content, then skip or delete","Write to unique per-attempt names and finalize with a metadata step instead of relying on overwrite"],"exampleFix":"// before\nfs.rename(new Path(\"/staging/part-1\"), new Path(\"/warehouse/db\")); // /warehouse/db/part-1 already exists as a file\n\n// after\nPath target = new Path(new Path(\"/warehouse/db\"), \"part-1\");\nif (fs.exists(target)) {\n  fs.delete(target, false);\n}\nfs.rename(new Path(\"/staging/part-1\"), new Path(\"/warehouse/db\"));","handlingStrategy":"validation","validationCode":"Path effectiveTarget = dst;\nif (fs.exists(dst) && fs.getFileStatus(dst).isDirectory()) {\n  effectiveTarget = new Path(dst, src.getName());\n}\nif (fs.exists(effectiveTarget) && fs.getFileStatus(effectiveTarget).isFile()) {\n  fs.delete(effectiveTarget, false); // deliberate overwrite\n}","typeGuard":null,"tryCatchPattern":"try {\n  fs.rename(src, dst);\n} catch (RenameFailedException e) {\n  if (e.getMessage().contains(\"new destination is an existed file\")) {\n    // delete dst/<srcname> if overwrite is intended, then retry once\n  } else {\n    throw e;\n  }\n}","preventionTips":["Check dst/<srcname> with exists() before every move-into-directory","Make commit steps idempotent so a re-run sees the existing target as done","Use unique per-run names when overwriting is not acceptable"],"tags":["hadoop","obs","rename","destination-exists","overwrite"],"backgroundTag":"rename-destination-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}