apache/hadoop · error · PathIsNotDirectoryException

Is not a directory

Error message

Is not a directory

What it means

PathIsNotDirectoryException ('Is not a directory') thrown by Mkdir.processPath (Mkdir.java:65) when 'hadoop fs -mkdir' targets an existing path that is a regular FILE. mkdir can only create/reuse directories; a colliding file name is always an error, even with -p (the -p flag only relaxes the existing-DIRECTORY case, not a file collision).

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Mkdir.java:65

    "-p: Do not fail if the directory already exists";

  private boolean createParents;
  
  @Override
  protected void processOptions(LinkedList<String> args) {
    CommandFormat cf = new CommandFormat(1, Integer.MAX_VALUE, "p");
    cf.parse(args);
    createParents = cf.getOpt("p");
  }

  @Override
  protected void processPath(PathData item) throws IOException {
    if (item.stat.isDirectory()) {
      if (!createParents) {
        throw new PathExistsException(item.toString());
      }
    } else {
      throw new PathIsNotDirectoryException(item.toString());
    }
  }

  @Override
  protected void processNonexistentPath(PathData item) throws IOException {
    if (!createParents) {
      // check if parent exists. this is complicated because getParent(a/b/c/) returns a/b/c, but
      // we want a/b
      final Path itemPath = new Path(item.path.toString());
      final Path itemParentPath = itemPath.getParent();

      if(itemParentPath == null) {
        throw new PathNotFoundException(String.format(
            "Item: %s parent's path is null. This can happen if mkdir is " +
                "called on root, so there's no parent.", itemPath.toString()));
      }

      if (!item.fs.exists(itemParentPath)) {

View on GitHub (pinned to 2add963021)

Solutions

1. Pick a different directory name that does not collide with the file
2. Move or delete the file if it is stale: 'hadoop fs -rm /data/file.txt' then mkdir
3. Audit the producer that created a file where your layout expects a directory and fix its path convention

Example fix

# before
hadoop fs -mkdir -p /topic/events    # /topic/events already exists as a FILE -> Is not a directory

# after
hadoop fs -mv /topic/events /topic/events.bak
hadoop fs -mkdir -p /topic/events

if (fs.exists(path)) {
  FileStatus st = fs.getFileStatus(path);
  if (!st.isDirectory()) {
    throw new PathIsNotDirectoryException(path.toString());
  }
}

static boolean canCreateDirectoryAt(FileSystem fs, Path p) throws IOException {
  return !fs.exists(p) || fs.getFileStatus(p).isDirectory();
}

try {
  fs.mkdirs(path);
} catch (PathIsNotDirectoryException e) {
  // a FILE occupies the path: rename/remove it, or choose another name
}

When it happens

Trigger: 'hadoop fs -mkdir /data/file.txt' where file.txt is an existing file; a flat-file naming convention that later becomes a directory path (e.g. /topic/name as file, then /topic/name/part-0 expected); 'mkdir -p' on an existing file hits this same branch.

Common situations: Schema evolution in data lakes: a path first used as a file marker, later expected as a directory; scripts choosing output paths that collide with a _SUCCESS-style file; case-insensitive filesystems colliding with differently-cased files.

Related errors

• Is not a directory
• Item: %s parent's path is null. This can happen if mkdir is
• Illegal option {}
• Not enough arguments: expected {} but got {}
• Too many arguments: expected {} but got {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5ab722da38761ef9. Report an issue: GitHub.