What causes "Mkdirs failed to create {}" in apache/hadoop?

Inspect the path named in the message: create it manually with `mkdir -p ` to surface the real OS error (permission denied, not a directory, read-only FS)

apache/hadoop · error · IOException

Mkdirs failed to create {}

Error message

Mkdirs failed to create {}

What it means

RunJar, the class behind the `hadoop jar` command, unpacks a job jar into a working directory before invoking its main class. ensureDirectory() calls File.mkdirs() on that directory; if mkdirs() returns false AND the path is not already a directory, it throws IOException("Mkdirs failed to create <dir>"). The job jar never runs because its unpack target cannot be created — almost always a permission conflict, a regular file occupying the path, or a full/read-only filesystem.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/RunJar.java:242

        }
      }
      if (numOfFailedLastModifiedSet > 0) {
        LOG.warn("Could not set last modfied time for {} file(s)",
            numOfFailedLastModifiedSet);
      }
    }
  }

  /**
   * Ensure the existence of a given directory.
   *
   * @param dir Directory to check
   *
   * @throws IOException if it cannot be created and does not already exist
   */
  private static void ensureDirectory(File dir) throws IOException {
    if (!dir.mkdirs() && !dir.isDirectory()) {
      throw new IOException("Mkdirs failed to create " +
                            dir.toString());
    }
  }

  /** Run a Hadoop job jar.  If the main class is not in the jar's manifest,
   * then it must be provided on the command line.
   *
   * @param args args.
   * @throws Throwable error.
   */
  public static void main(String[] args) throws Throwable {
    new RunJar().run(args);
  }

  public void run(String[] args) throws Throwable {
    String usage = "RunJar jarFile [mainClass] args...";

    if (args.length < 1) {

View on GitHub (pinned to 2add963021)

Solutions

Inspect the path named in the message: create it manually with `mkdir -p <dir>` to surface the real OS error (permission denied, not a directory, read-only FS)
If a regular file sits at that path, delete or rename it
Give the launcher a writable temp area: run with -Djava.io.tmpdir=/var/tmp/$USER (and/or set hadoop.tmp.dir to a writable location)
Check the filesystem: `df -h <dir>` for space and `touch <dir>/probe` for writability
Re-run `hadoop jar` as a user with write access to the temp directory

Example fix

# before: fails with Mkdirs failed to create /tmp/hadoop-unjar123
hadoop jar app.jar com.example.Main
# diagnose the real error
ls -ld /tmp/hadoop-unjar123; mkdir -p /tmp/hadoop-unjar123
# after: point RunJar at a writable temp dir
hadoop jar app.jar com.example.Main -Djava.io.tmpdir=/var/tmp/$USER  # or: export HADOOP_OPTS="$HADOOP_OPTS -Djava.io.tmpdir=/var/tmp/$USER"

Defensive patterns

Strategy: validation

Validate before calling

File tmp = new File(System.getProperty("java.io.tmpdir"));
File unjar = new File(tmp, "hadoop-unjar");
if (unjar.exists() && !unjar.isDirectory())
  throw new IOException("path occupied by a regular file: " + unjar);
if (!unjar.exists() && !unjar.mkdirs())
  throw new IOException("cannot create " + unjar + " — check permissions/disk");

Try / catch

try {
  new RunJar().run(args);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Mkdirs failed to create")) {
    // report the named directory; fix permissions/occupying file and relaunch
  }
  throw e;
}

Prevention

Give the hadoop-launching user a writable temp directory policy (java.io.tmpdir / hadoop.tmp.dir)
Clean stale unjar directories after crashed runs
In containers, mount writable emptyDir/tmpfs at the temp location

When it happens

Trigger: Running `hadoop jar app.jar ...` when the unjar working directory (created under java.io.tmpdir / the temp area) cannot be created: parent directory lacks write permission for the launching user, a regular file already exists at the exact directory path, the filesystem is full, read-only (container overlayfs), or SELinux/NFS root-squash denies the mkdir.

Common situations: Running as an unprivileged user against a locked-down /tmp; a leftover file from a previously crashed run occupying the target path; Docker/Kubernetes containers with read-only root filesystems; disk exhaustion during CI; NFS mounts with squashed privileges.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/2bbd92d3e519426c. Report an issue: GitHub.