apache/hadoop · error · ParentNotDirectoryException

parent is not a dir

Error message

parent is not a dir

What it means

The second precondition in primitiveMkdir(createParent=false): the parent path exists but is a regular file, so stat.isDirectory() is false and ParentNotDirectoryException('parent is not a dir') is thrown. A file cannot act as a directory, so creating f under it is impossible.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:1405

   * @param f the path.
   * @param absolutePermission permission.
   * @param createParent create parent.
   * @throws IOException IO failure.
   */
  @Deprecated
  protected void primitiveMkdir(Path f, FsPermission absolutePermission,
                    boolean createParent)
    throws IOException {

    if (!createParent) { // parent must exist.
      // since the this.mkdirs makes parent dirs automatically
      // we must throw exception if parent does not exist.
      final FileStatus stat = getFileStatus(f.getParent());
      if (stat == null) {
        throw new FileNotFoundException("Missing parent:" + f);
      }
      if (!stat.isDirectory()) {
        throw new ParentNotDirectoryException("parent is not a dir");
      }
      // parent does exist - go ahead with mkdir of leaf
    }
    // Default impl is to assume that permissions do not matter and hence
    // calling the regular mkdirs is good enough.
    // FSs that implement permissions should override this.
    if (!this.mkdirs(f, absolutePermission)) {
      throw new IOException("mkdir of "+ f + " failed");
    }
  }

  /**
   * Opens an FSDataOutputStream at the indicated Path with write-progress
   * reporting. Same as create(), except fails if parent directory doesn't
   * already exist.
   * @param f the file name to open
   * @param overwrite if a file with this name already exists, then if true,
   * the file will be overwritten, and if false an error will be thrown.

View on GitHub (pinned to 2add963021)

Solutions

1. If the file is disposable, delete it (fs.delete(parent, true)) and recreate the directory
2. Otherwise choose a different destination layout so files and directories never share prefixes
3. Add a pre-flight check that every prefix of f is either absent or a directory

Example fix

// before
fs.primitiveMkdir(new Path("/out/part-0/data"), perm, false); // /out/part-0 is a FILE

// after
Path parent = new Path("/out/part-0/data").getParent();
if (fs.exists(parent) && !fs.getFileStatus(parent).isDirectory()) {
  fs.delete(parent, true);
}
fs.mkdirs(parent);

Path parent = f.getParent();
if (fs.exists(parent) && !fs.getFileStatus(parent).isDirectory()) {
  throw new IllegalStateException("Path prefix occupied by a file: " + parent);
}
fs.primitiveMkdir(f, perm, false);

When it happens

Trigger: primitiveMkdir(f, perm, false) where f.getParent() resolves to an existing file - e.g. creating /a/b/c while /a/b is a file.

Common situations: Stale output files occupying a directory prefix from earlier runs; layouts that changed between file-at-path and directory-at-path; partially cleaned temp trees.

Related errors

• Mkdirs failed to create {} (exists={}, cwd={})
• Missing parent:${f}
• mkdir of ${f} failed
• rename destination parent ${parent} is a file.
• {} already exists

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/591728943e7e861c. Report an issue: GitHub.