apache/hadoop · error · IOException

Mkdirs failed to create ${dir}

Error message

Mkdirs failed to create ${dir}

What it means

The exists-or-create variant in FileIoProvider: it computes succeeded = dir.isDirectory() || dir.mkdirs(), and throws IOException('Mkdirs failed to create ' + dir) when both fail. Same failure surface as the plain mkdirs wrapper — the path is not a directory and could not be created — used by callers that only need the directory to exist.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FileIoProvider.java:697

   * @param volume  target volume. null if unavailable.
   * @param dir  directory to be created.
   * @throws IOException  if the directory could not created
   */
  public void mkdirsWithExistsCheck(
      @Nullable FsVolumeSpi volume, File dir) throws IOException {
    final long begin = profilingEventHook.beforeMetadataOp(volume, MKDIRS);
    boolean succeeded = false;
    try {
      faultInjectorEventHook.beforeMetadataOp(volume, MKDIRS);
      succeeded = dir.isDirectory() || dir.mkdirs();
      profilingEventHook.afterMetadataOp(volume, MKDIRS, begin);
    } catch(Exception e) {
      onFailure(volume, begin);
      throw e;
    }

    if (!succeeded) {
      throw new IOException("Mkdirs failed to create " + dir);
    }
  }

  /**
   * Get a listing of the given directory using
   * {@link FileUtil#listFiles(File)}.
   *
   * @param volume  target volume. null if unavailable.
   * @param dir  Directory to be listed.
   * @return  array of file objects representing the directory entries.
   * @throws IOException
   */
  public File[] listFiles(
      @Nullable FsVolumeSpi volume, File dir) throws IOException {
    final long begin = profilingEventHook.beforeMetadataOp(volume, LIST);
    try {
      faultInjectorEventHook.beforeMetadataOp(volume, LIST);
      File[] children = FileUtil.listFiles(dir);

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the path is not occupied by a plain file and remove the conflict if it is.
2. Fix parent ownership/permissions for the DataNode user (chown/chmod on the data-dir tree).
3. Check `df -h` and `df -i` on the volume; free space or inodes.
4. Validate the mount health if the volume is network-backed.

Example fix

// before
fileIoProvider.mkdirsWithExistsCheck(volume, dir);

// after: distinguish 'already a dir' from 'occupied by a file' up front
if (!dir.isDirectory() && dir.exists()) {
  throw new IOException("cannot create directory, path is a file: " + dir);
}
fileIoProvider.mkdirsWithExistsCheck(volume, dir);

if (!dir.isDirectory() && dir.exists()) {
  // path is occupied by a non-directory: resolve the conflict first
}
if (dir.getParentFile() != null && !dir.getParentFile().canWrite()) {
  // parent not writable by the DataNode user: fix permissions
}

try {
  fileIoProvider.mkdirsWithExistsCheck(volume, dir);
} catch (IOException e) {
  // dir named in the message: check path conflicts, perms, and volume space/inodes
}

When it happens

Trigger: Calling this ensure-directory-exists helper when the path is occupied by a regular file, a parent is missing or not writable by the DataNode user, the volume is full (blocks or inodes), or the storage backing the volume (NFS/fuse) rejects the mkdir.

Common situations: DataNode data-dir permission drift after a user change or restore; inode exhaustion on small local filesystems; leftover files at directory paths after an interrupted operation; degraded NFS mounts.

Related errors

• Mkdirs failed to create {}
• Cannot create directory {}
• Cannot remove directory {}
• Cannot remove directory {}
• Failed to move block file for ${this} from ${srcfile} to ${d

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/2fe64a97583c927e. Report an issue: GitHub.