{"record":{"id":"4b519eab4e3f027a","repo":"apache/hadoop","slug":"destination-parent-is-not-a-directory-4b519e","errorCode":null,"errorMessage":"destination parent is not a directory","messagePattern":"destination parent is not a directory","errorType":"exception","errorClass":"RenameFailedException","httpStatus":null,"severity":"warning","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java","lineNumber":2498,"sourceCode":"      }\n\n    } catch (FileNotFoundException e) {\n      LOG.debug(\"rename: destination path {} not found\", dst);\n      // Parent must exist\n      Path parent = dst.getParent();\n      if (!pathToKey(parent).isEmpty()\n          && !parent.equals(src.getParent())) {\n        try {\n          // make sure parent isn't a file.\n          // don't look for parent being a dir as there is a risk\n          // of a race between dest dir cleanup and rename in different\n          // threads.\n          S3AFileStatus dstParentStatus = innerGetFileStatus(parent,\n              false, StatusProbeEnum.FILE);\n          // if this doesn't raise an exception then\n          // the parent is a file or a dir.\n          if (!dstParentStatus.isDirectory()) {\n            throw new RenameFailedException(src, dst,\n                \"destination parent is not a directory\");\n          }\n        } catch (FileNotFoundException expected) {\n          // nothing was found. Don't worry about it;\n          // expect rename to implicitly create the parent dir\n        }\n      }\n    }\n    return Pair.of(srcStatus, dstStatus);\n  }\n\n  /**\n   * The inner rename operation. See {@link #rename(Path, Path)} for\n   * the description of the operation.\n   * This operation throws an exception on any failure which needs to be\n   * reported and downgraded to a failure.\n   * Retries: retry translated, assuming all operations it is called do\n   * so. For safely, consider catch and handle SdkException","sourceCodeStart":2480,"sourceCodeEnd":2516,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2480-L2516","documentation":"When the destination does not exist (the FileNotFoundException branch), initiateRename() checks dst's parent: if the parent resolves to something that is not a directory (i.e. an object), RenameFailedException 'destination parent is not a directory' is thrown with the default exit code, so rename() logs it at INFO and returns false. A missing parent is explicitly fine - S3A creates intermediate directories implicitly; the source notes the dir/file race only affects marker interpretation.","triggerScenarios":"rename(src, dst) where some ancestor of dst is an existing object, e.g. dst = a/b/c while a/b exists as a file.","commonSituations":"Deep destination trees where an earlier step wrote a file at what is now expected to be a directory level; path-layout changes between runs; generated destination paths colliding with old file outputs.","solutions":["Delete the object occupying the ancestor path; the rename then recreates the directory chain","Correct the destination path construction","Validate the parent chain with fs.getFileStatus(dst.getParent()) before renaming"],"exampleFix":"// before: fails (returns false) if a/b exists as a file\nfs.rename(src, new Path(\"a/b/c\"));\n\n// after: clear file-vs-directory conflicts in the ancestor chain\nPath parent = dst.getParent();\nwhile (parent != null && !parent.isRoot()) {\n  if (fs.exists(parent) && fs.getFileStatus(parent).isFile()) {\n    fs.delete(parent, false);\n  }\n  parent = parent.getParent();\n}\nfs.rename(src, dst);","handlingStrategy":"validation","validationCode":"static void ensureParentIsDirectory(FileSystem fs, Path dst) throws IOException {\n  Path parent = dst.getParent();\n  if (parent == null || parent.isRoot()) return;\n  if (fs.exists(parent) && !fs.getFileStatus(parent).isDirectory()) {\n    throw new IOException(\"ancestor of destination is a file: \" + parent);\n  }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate the ancestor chain of generated destination paths","Remove file/directory conflicts from previous layouts during migration","Remember missing parents are fine (auto-created); only existing file ancestors fail, and rename returns false"],"tags":["s3a","rename","parent-not-directory","path-conflict"],"backgroundTag":"parent-not-a-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}