{"record":{"id":"48b983a5c725743f","repo":"apache/hadoop","slug":"new-destination-is-an-existed-directory","errorCode":null,"errorMessage":"new destination is an existed directory","messagePattern":"new destination is an existed directory","errorType":"exception","errorClass":"RenameFailedException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSObjectBucketUtils.java","lineNumber":127,"sourceCode":"\n    FileStatus dstStatus;\n    try {\n      dstStatus = owner.getFileStatus(dst);\n      // if there is no destination entry, an exception is raised.\n      // hence this code sequence can assume that there is something\n      // at the end of the path; the only detail being what it is and\n      // whether or not it can be the destination of the rename.\n      if (dstStatus.isDirectory()) {\n        String newDstKey = OBSCommonUtils.maybeAddTrailingSlash(dstKey);\n        String filename = srcKey.substring(\n            OBSCommonUtils.pathToKey(owner, src.getParent()).length()\n                + 1);\n        newDstKey = newDstKey + filename;\n        dstKey = newDstKey;\n        dstStatus = owner.getFileStatus(\n            OBSCommonUtils.keyToPath(dstKey));\n        if (dstStatus.isDirectory()) {\n          throw new RenameFailedException(src, dst,\n              \"new destination is an existed directory\")\n              .withExitCode(false);\n        } else {\n          throw new RenameFailedException(src, dst,\n              \"new destination is an existed file\")\n              .withExitCode(false);\n        }\n      } else {\n\n        if (srcKey.equals(dstKey)) {\n          LOG.warn(\n              \"rename: src and dest refer to the same file or\"\n                  + \" directory: {}\",\n              dst);\n          return true;\n        } else {\n          throw new RenameFailedException(src, dst,\n              \"destination is an existed file\")","sourceCodeStart":109,"sourceCodeEnd":145,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSObjectBucketUtils.java#L109-L145","documentation":"Under Hadoop rename semantics, when the destination exists and is a directory the source is moved inside it as dst/<srcname>. renameBasedOnObject builds that key, stats it, and when it resolves to an existing directory throws RenameFailedException('new destination is an existed directory').withExitCode(false). The false exit code marks the failure as permanent, not retryable.","triggerScenarios":"fs.rename('/src/d1', '/dst') when /dst/d1 already exists as a directory; concurrent jobs moving different sources into the same destination directory with colliding names; re-running a move after a first attempt already created the target directory.","commonSituations":"Date-partitioned layouts where a partition directory is moved into an existing tree; ingestion jobs that mkdirs the target first (which silently succeeds on OBS) and then rename into it; non-idempotent commit logic re-executed after a partial failure.","solutions":["Check whether dst/<srcname> exists before the rename and delete or merge it deliberately","Skip the rename when the target already holds the correct content, making the job idempotent","Use a unique destination name (timestamp or run id) when name collisions are possible"],"exampleFix":"// before\nfs.rename(new Path(\"/staging/part-1\"), new Path(\"/warehouse/db\")); // /warehouse/db/part-1 already exists as a dir\n\n// after\nPath target = new Path(new Path(\"/warehouse/db\"), \"part-1\");\nif (fs.exists(target)) {\n  fs.delete(target, true); // or skip if already ingested\n}\nfs.rename(new Path(\"/staging/part-1\"), new Path(\"/warehouse/db\"));","handlingStrategy":"validation","validationCode":"Path effectiveTarget = dst;\nif (fs.exists(dst) && fs.getFileStatus(dst).isDirectory()) {\n  effectiveTarget = new Path(dst, src.getName());\n}\nif (fs.exists(effectiveTarget) && fs.getFileStatus(effectiveTarget).isDirectory()) {\n  throw new IOException(\"Target already exists as a directory: \" + effectiveTarget);\n}","typeGuard":null,"tryCatchPattern":"try {\n  fs.rename(src, dst);\n} catch (RenameFailedException e) {\n  if (e.getMessage().contains(\"new destination is an existed directory\")) {\n    // permanent failure: clean or merge dst/<srcname>, then decide\n  } else {\n    throw e;\n  }\n}","preventionTips":["Compute the effective destination (dst + src name) and check it before rename","Design moves to be idempotent: verify target content instead of blind rename","Avoid two jobs moving data into the same destination directory","Do not retry RenameFailedException: withExitCode(false) marks it permanent"],"tags":["hadoop","obs","rename","destination-exists","directory"],"backgroundTag":"rename-destination-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}