apache/hadoop · error · PathExistsException

File exists

Error message

File exists

What it means

PathExistsException ('File exists') thrown by Mkdir.processPath (Mkdir.java:62) when 'hadoop fs -mkdir' (without -p) is called on a path that already exists and is a directory. The non-parents mkdir is POSIX-strict: an existing directory is an error. With -p (createParents) the same situation is a no-op success.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Mkdir.java:62

  public static final String USAGE = "[-p] <path> ...";
  public static final String DESCRIPTION =
    "Create a directory in specified location.\n" +
    "-p: Do not fail if the directory already exists";

  private boolean createParents;
  
  @Override
  protected void processOptions(LinkedList<String> args) {
    CommandFormat cf = new CommandFormat(1, Integer.MAX_VALUE, "p");
    cf.parse(args);
    createParents = cf.getOpt("p");
  }

  @Override
  protected void processPath(PathData item) throws IOException {
    if (item.stat.isDirectory()) {
      if (!createParents) {
        throw new PathExistsException(item.toString());
      }
    } else {
      throw new PathIsNotDirectoryException(item.toString());
    }
  }

  @Override
  protected void processNonexistentPath(PathData item) throws IOException {
    if (!createParents) {
      // check if parent exists. this is complicated because getParent(a/b/c/) returns a/b/c, but
      // we want a/b
      final Path itemPath = new Path(item.path.toString());
      final Path itemParentPath = itemPath.getParent();

      if(itemParentPath == null) {
        throw new PathNotFoundException(String.format(
            "Item: %s parent's path is null. This can happen if mkdir is " +
                "called on root, so there's no parent.", itemPath.toString()));

View on GitHub (pinned to 2add963021)

Solutions

1. Use -p for idempotent creation: 'hadoop fs -mkdir -p /data/exists' succeeds if the dir is already there
2. If a fresh directory is required, 'hadoop fs -rm -r /data/exists' first (accepting data loss), then mkdir
3. Pre-check with 'hadoop fs -test -d /path' when you must distinguish created vs already-existed

Example fix

# before
hadoop fs -mkdir /staging/daily    # File exists on re-run

# after
hadoop fs -mkdir -p /staging/daily

if (fs.exists(path) && fs.getFileStatus(path).isDirectory()) {
  // already exists: use mkdir -p semantics (no-op) instead of failing
}

static boolean mkdirNeeded(FileSystem fs, Path p) throws IOException {
  return !fs.exists(p);
}

try {
  fs.mkdirs(path);
} catch (PathExistsException e) {
  // directory already present; treat as success for idempotent provisioning
}

When it happens

Trigger: 'hadoop fs -mkdir /data/exists' where /data/exists is already a directory; idempotent re-runs of provisioning scripts that do not use -p; race where two jobs mkdir the same staging path simultaneously.

Common situations: Deployment pipelines that provision the same directory each run; make-style targets without guards; onboarding docs using bare mkdir where -p was intended.

Related errors

• Is not a directory
• Item: %s parent's path is null. This can happen if mkdir is
• Can't make directory for path '%s' since it is a file.
• Cannot create directories because of existing file: %s
• Mkdirs failed to create {} (exists={}, cwd={})

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1c8494368b46b50e. Report an issue: GitHub.