apache/hadoop · error · IOException

Mkdirs failed to create {}

Error message

Mkdirs failed to create {}

What it means

FileIoProvider.mkdirs(volume, dir) calls java.io.File.mkdirs(); if it returns false and the path is still not a directory, an IOException('Mkdirs failed to create ' + dir) is thrown. File.mkdirs() returns false when a parent is missing or non-writable, a non-directory file already occupies the path, or the filesystem refuses the operation (ENOSPC, EIO, read-only mount).

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FileIoProvider.java:670

   *                     not exist and could not be created.
   */
  public boolean mkdirs(
      @Nullable FsVolumeSpi volume, File dir) throws IOException {
    final long begin = profilingEventHook.beforeMetadataOp(volume, MKDIRS);
    boolean created = false;
    boolean isDirectory;
    try {
      faultInjectorEventHook.beforeMetadataOp(volume, MKDIRS);
      created = dir.mkdirs();
      isDirectory = !created && dir.isDirectory();
      profilingEventHook.afterMetadataOp(volume, MKDIRS, begin);
    } catch(Exception e) {
      onFailure(volume, begin);
      throw e;
    }

    if (!created && !isDirectory) {
      throw new IOException("Mkdirs failed to create " + dir);
    }
    return created;
  }

  /**
   * Create the target directory using {@link File#mkdirs()} only if
   * it doesn't exist already.
   *
   * @param volume  target volume. null if unavailable.
   * @param dir  directory to be created.
   * @throws IOException  if the directory could not created
   */
  public void mkdirsWithExistsCheck(
      @Nullable FsVolumeSpi volume, File dir) throws IOException {
    final long begin = profilingEventHook.beforeMetadataOp(volume, MKDIRS);
    boolean succeeded = false;
    try {
      faultInjectorEventHook.beforeMetadataOp(volume, MKDIRS);

View on GitHub (pinned to 2add963021)

Solutions

1. Check ownership and permissions on the target dir and all parents: chown -R <dn-user>:<dn-group> and chmod so the DataNode user can write.
2. Check disk and inode headroom on the volume: `df -h <vol>` and `df -i <vol>`; free space if either is exhausted.
3. Confirm nothing occupies the path as a plain file: `ls -ld <dir>`; remove or relocate the conflicting file.
4. If the volume sits on NFS/fuse, verify the mount is healthy (`mount`, dmesg) before retrying.

Example fix

// before: assume mkdirs succeeds
fileIoProvider.mkdirs(volume, dir);

// after: pre-flight the conditions that make mkdirs fail
Path p = dir.toPath();
if (Files.exists(p) && !Files.isDirectory(p)) {
  throw new IOException("path occupied by a non-directory file: " + dir);
}
if (!Files.isWritable(p.getParent())) {
  throw new IOException("parent not writable by " + System.getProperty("user.name"));
}
fileIoProvider.mkdirs(volume, dir);

Path p = dir.toPath();
if (Files.exists(p) && !Files.isDirectory(p)) {
  // a file occupies the path: remove or relocate it first
}
if (!Files.isWritable(p.getParent())) {
  // fix ownership/permissions for the DataNode user before mkdirs
}
if (Files.getFileStore(p.getParent()).getUsableSpace() <= 0) {
  // no space on volume: free some before mkdirs
}

try {
  fileIoProvider.mkdirs(volume, dir);
} catch (IOException e) {
  // message names the dir; check perms, path conflicts, and df -h / df -i on that volume
}

When it happens

Trigger: The DataNode creating a block-pool subdirectory or block subdir on a volume where a parent directory lacks write permission for the DN user, a regular file already exists at the path, the disk or inode table is full, or the underlying mount (NFS/fuse) errors out.

Common situations: DataNode started under a different user than the owner of dfs.datanode.data.dir; disk full or inode exhaustion; a stray file occupying a directory path; flaky fuse/NFS mounts backing a data dir.

Related errors

• Mkdirs failed to create ${dir}
• Cannot create directory {}
• Cannot remove directory {}
• Cannot remove directory {}
• Failed to move block file for ${this} from ${srcfile} to ${d

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6b53a0241a4382d3. Report an issue: GitHub.