apache/hadoop · error · FileAlreadyExistsException

Destination exists and is not a directory: " + p2f.getCanoni

Error message

Destination exists and is not a directory: " + p2f.getCanonicalPath()

What it means

mkdirsWithOptionalPermission throws FileAlreadyExistsException('Destination exists and is not a directory') when the final target path itself exists as a regular file. mkdirs refuses to overwrite an existing file with a directory. The message reports the canonical path, which resolves symlinks — useful when the visible path differs from the real blocker.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:916

  }

  private boolean mkdirsWithOptionalPermission(Path f, FsPermission permission)
      throws IOException {
    if(f == null) {
      throw new IllegalArgumentException("mkdirs path arg is null");
    }
    Path parent = f.getParent();
    File p2f = pathToFile(f);
    File parent2f = null;
    if(parent != null) {
      parent2f = pathToFile(parent);
      if(parent2f != null && parent2f.exists() && !parent2f.isDirectory()) {
        throw new ParentNotDirectoryException("Parent path is not a directory: "
            + parent);
      }
    }
    if (p2f.exists() && !p2f.isDirectory()) {
      throw new FileAlreadyExistsException("Destination exists" +
              " and is not a directory: " + p2f.getCanonicalPath());
    }
    return (parent == null || parent2f.exists() || mkdirs(parent)) &&
      (mkOneDirWithMode(f, p2f, permission) || p2f.isDirectory());
  }
  
  
  @Override
  public Path getHomeDirectory() {
    return this.makeQualified(new Path(System.getProperty("user.home")));
  }

  /**
   * Set the working directory to the given directory.
   */
  @Override
  public void setWorkingDirectory(Path newDir) {
    workingDir = makeAbsolute(newDir);

View on GitHub (pinned to 2add963021)

Solutions

1. Check the canonical path in the message with ls -l; confirm it is a regular file or a symlink to one.
2. Delete or move the file, then retry mkdirs.
3. Disambiguate the layout: use distinct names for files vs. directories (e.g. /data/daily/ for dirs, /data/daily.dat for files).
4. Pre-flight in code: if (fs.exists(p) && !fs.getFileStatus(p).isDirectory()) handle the conflict explicitly.

Example fix

// before
Path daily = new Path("/data/daily"); // exists as a regular file
fs.mkdirs(daily); // FileAlreadyExistsException

// after
Path daily = new Path("/data/daily");
if (fs.exists(daily) && !fs.getFileStatus(daily).isDirectory()) {
  fs.rename(daily, new Path("/data/daily.old-" + System.currentTimeMillis()));
}
fs.mkdirs(daily);

if (fs.exists(p)) {
  if (!fs.getFileStatus(p).isDirectory()) {
    throw new FileAlreadyExistsException(
        p + " exists as a file; refusing to replace with directory");
  }
} else {
  fs.mkdirs(p);
}

try {
  fs.mkdirs(p);
} catch (FileAlreadyExistsException e) {
  // message contains the canonical path; inspect and resolve the file conflict
  throw new IOException("File occupies directory target " + p
      + "; delete or relocate it", e);
}

When it happens

Trigger: Calling fs.mkdirs("/data/daily") when /data/daily is a regular file (e.g. created by a previous run's create() or a redirect target), or a symlink at that path pointing to a file.

Common situations: Job output layouts switching between file-per-day and directory-per-day, leftover files from prior runs under temp dirs, shell redirections (cmd > /data/daily) having created a file where code now wants a directory.

Related errors

• Parent path is not a directory: " + parent
• Mkdirs failed to create {} (exists={}, cwd={})
• File already exists: " + f
• Mkdirs failed to create " + parent.toString()
• mkdirs path arg is null

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0017d98bb52c61c5. Report an issue: GitHub.