apache/hadoop · error · FileAlreadyExistsException

Path is not a directory: {}

Error message

Path is not a directory: {}

What it means

FileAlreadyExistsException from FSDirMkdirOp.mkdirs when the target path's final component already exists as a file - HDFS refuses to mkdir over a file. An existing directory at the same path is fine (mkdir succeeds as a no-op); only a file at the final component collides.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirMkdirOp.java:57

import java.util.Optional;

import static org.apache.hadoop.util.Time.now;

class FSDirMkdirOp {

  static FileStatus mkdirs(FSNamesystem fsn, FSPermissionChecker pc, String src,
      PermissionStatus permissions, boolean createParent) throws IOException {
    FSDirectory fsd = fsn.getFSDirectory();
    if(NameNode.stateChangeLog.isDebugEnabled()) {
      NameNode.stateChangeLog.debug("DIR* NameSystem.mkdirs: " + src);
    }
    fsd.writeLock();
    try {
      INodesInPath iip = fsd.resolvePath(pc, src, DirOp.CREATE);

      final INode lastINode = iip.getLastINode();
      if (lastINode != null && lastINode.isFile()) {
        throw new FileAlreadyExistsException("Path is not a directory: " + src);
      }

      if (lastINode == null) {
        if (fsd.isPermissionEnabled()) {
          fsd.checkAncestorAccess(pc, iip, FsAction.WRITE);
        }

        if (!createParent) {
          fsd.verifyParentDir(iip);
        }

        // validate that we have enough inodes. This is, at best, a
        // heuristic because the mkdirs() operation might need to
        // create multiple inodes.
        fsn.checkFsObjectLimit();

        // Ensure that the user can traversal the path by adding implicit
        // u+wx permission to all ancestor directories.

View on GitHub (pinned to 2add963021)

Solutions

1. Remove or rename the conflicting file: `hdfs dfs -rm <path>` (or `-mv` it aside), then mkdir.
2. Or write to a fresh path (append a run id) when the old file must be kept.
3. Pre-check in code: only call mkdirs when the path is absent or already a directory.

Example fix

// before
fs.mkdirs(new Path("/jobs/run42")); // FileAlreadyExistsException: /jobs/run42 is a file

// after
Path p = new Path("/jobs/run42");
if (fs.exists(p) && !fs.getFileStatus(p).isDirectory()) {
  fs.delete(p, false);            // or pick /jobs/run42_r2
}
fs.mkdirs(p);

if (fs.exists(p) && !fs.getFileStatus(p).isDirectory()) {
  throw new FileAlreadyExistsException("A file exists at " + p
      + "; remove it or choose a different path");
}
fs.mkdirs(p);

try {
  fs.mkdirs(p);
} catch (FileAlreadyExistsException e) {
  FileStatus st = fs.getFileStatus(p);
  if (!st.isDirectory()) {
    fs.delete(p, false); // only if disposable, then:
    fs.mkdirs(p);
  }
}

When it happens

Trigger: `hdfs dfs -mkdir /tmp/out` where /tmp/out is an existing file (e.g., a former _SUCCESS-style marker or previous output file); `mkdir -p` where the last component is a file; job restart writing to a path now occupied by a file.

Common situations: MapReduce/Spark output directories that were previously written as files; a rename leaving a file on the intended directory path; shared scratch/temp paths reused across services.

Related errors

• Destination '{parentFile}' directory cannot be created
• {} already exists
• Can't make directory for path '%s' since it is a file.
• The source {src} and destination {dst} are the same
• {} doesn't support satisfyStoragePolicy

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0e3f2ae55b509060. Report an issue: GitHub.