apache/hadoop · error · FileAlreadyExistsException

<path> already exists as a directory

Error message

<path> already exists as a directory

What it means

resolvePathForStartFile rejects create/overwrite when the target path already resolves to a directory: a directory can never be replaced by a file, so FileAlreadyExistsException is thrown before lease or quota work begins. This mirrors POSIX open(O_CREAT) semantics where the target must be a regular file.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java:334

      // Create a temp node to findout the rack local nodes
      clientNode = new NodeBase(rName.get(0) + NodeBase.PATH_SEPARATOR_STR
          + clientMachine);
    }
    return clientNode;
  }

  static INodesInPath resolvePathForStartFile(FSDirectory dir,
      FSPermissionChecker pc, String src, EnumSet<CreateFlag> flag,
      boolean createParent) throws IOException {
    INodesInPath iip = dir.resolvePath(pc, src, DirOp.CREATE);
    if (dir.isPermissionEnabled()) {
      dir.checkAncestorAccess(pc, iip, FsAction.WRITE);
    }
    INode inode = iip.getLastINode();
    if (inode != null) {
      // Verify that the destination does not exist as a directory already.
      if (inode.isDirectory()) {
        throw new FileAlreadyExistsException(iip.getPath() +
            " already exists as a directory");
      }
      // Verifies it's indeed a file and perms allow overwrite
      INodeFile.valueOf(inode, src);
      if (dir.isPermissionEnabled() && flag.contains(CreateFlag.OVERWRITE)) {
        dir.checkPathAccess(pc, iip, FsAction.WRITE);
      }
    } else {
      if (!createParent) {
        dir.verifyParentDir(iip);
      }
      if (!flag.contains(CreateFlag.CREATE)) {
        throw new FileNotFoundException("Can't overwrite non-existent " + src);
      }
    }
    return iip;
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Point the writer at a file path that does not collide (for example outputPath/part-00000), or delete/rename the existing directory first
2. Guard with fs.exists(path) && fs.getFileStatus(path).isDirectory() before create
3. If a file was intended, remove the directory (after verifying it is safe) or choose another name

Example fix

// before: path exists as a directory
try (FSDataOutputStream out = fs.create(dirPath)) { ... }

// after: write a file inside it
Path outFile = new Path(dirPath, "part-00000");
if (fs.exists(dirPath) && fs.getFileStatus(dirPath).isDirectory()) {
  try (FSDataOutputStream out = fs.create(outFile, true)) { ... }
}

if (fs.exists(path) && fs.getFileStatus(path).isDirectory()) {
  throw new IllegalArgumentException("Output path is a directory: " + path);
}
try (FSDataOutputStream out = fs.create(path, true)) { ... }

static boolean isDirectoryTarget(FileSystem fs, Path p) throws IOException {
  return fs.exists(p) && fs.getFileStatus(p).isDirectory();
}

try {
  out = fs.create(path, true);
} catch (FileAlreadyExistsException e) {
  // path is a directory: pick a file path inside it or rename it first
}

When it happens

Trigger: FileSystem.create(path, ...) with or without overwrite where the resolved path is an existing directory; typical with output committers or frameworks writing to a path that collides with an existing directory.

Common situations: MapReduce/Spark/Hive output path equal to an existing directory; path-building code that accidentally targets an input directory; leftover directory from a previous run with a different layout.

Related errors

• no parent for {}
• Failed to delete {dir}
• Parent path is not a directory: {} {}
• Can't overwrite non-existent <src>
• <src> for client <clientMachine> already exists

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/2d7859befaef2b40. Report an issue: GitHub.