apache/hadoop · error · IOException

Mkdirs failed to create " + parent.toString()

Error message

Mkdirs failed to create " + parent.toString()

What it means

During create(), RawLocalFileSystem calls mkdirs(parent) and throws IOException('Mkdirs failed to create ...') when it returns false. The message names the parent directory. Failure means mkdirs could not create or find the parent as a directory: typically a permissions problem, a parent path occupied by a regular file (which surfaces deeper as ParentNotDirectoryException/FileAlreadyExistsException), or an unwritable ancestor.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:645

  }

  @Override
  public FSDataOutputStream create(Path f, boolean overwrite, int bufferSize,
    short replication, long blockSize, Progressable progress)
    throws IOException {
    return create(f, overwrite, true, bufferSize, replication, blockSize,
        progress, null);
  }

  private FSDataOutputStream create(Path f, boolean overwrite,
      boolean createParent, int bufferSize, short replication, long blockSize,
      Progressable progress, FsPermission permission) throws IOException {
    if (exists(f) && !overwrite) {
      throw new FileAlreadyExistsException("File already exists: " + f);
    }
    Path parent = f.getParent();
    if (parent != null && !mkdirs(parent)) {
      throw new IOException("Mkdirs failed to create " + parent.toString());
    }
    return new FSDataOutputStream(new BufferedIOStatisticsOutputStream(
        createOutputStreamWithMode(f, false, permission), bufferSize, true),
        statistics);
  }
  
  protected OutputStream createOutputStream(Path f, boolean append) 
      throws IOException {
    return createOutputStreamWithMode(f, append, null);
  }

  protected OutputStream createOutputStreamWithMode(Path f, boolean append,
      FsPermission permission) throws IOException {
    return new LocalFSFileOutputStream(f, append, permission);
  }
  
  @Override
  public FSDataOutputStream createNonRecursive(Path f, FsPermission permission,

View on GitHub (pinned to 2add963021)

Solutions

1. Inspect the exact parent path from the message: ls -l the ancestors and confirm each is a directory and writable by the process user.
2. If a regular file occupies the parent path, remove or relocate it: rm /var/data/out (a file) then retry.
3. Fix permissions/ownership: chown/chmod the intended parent chain or move the output under a writable base (e.g. under the user's home or ${hadoop.tmp.dir}).
4. For read-only or full mounts, relocate output to a writable filesystem and free space.

Example fix

# before (shell)
$ ls -l /var/data
-rw-r--r-- 1 root root 0 out      # regular file blocks the directory

# after
$ sudo rm /var/data/out
$ sudo mkdir -p /var/data/out && sudo chown $USER /var/data/out

// pre-flight the parent chain before create()
Path parent = p.getParent();
if (parent != null && !fs.exists(parent)) {
  FileStatus ancestor = firstExistingAncestor(fs, parent);
  if (!ancestor.isDirectory()) {
    throw new IOException("Ancestor " + ancestor.getPath() + " is a file");
  }
  // writable check
  if (!fs.getFileStatus(ancestor.getPath()).getPermission()
        .getUserAction().implies(FsAction.WRITE)) {
    LOG.warn("{} may not be writable by this user", ancestor.getPath());
  }
}
FSDataOutputStream out = fs.create(p, true);

try {
  return fs.create(p, overwrite);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Mkdirs failed")) {
    throw new IOException("Cannot create parent of " + p
        + ": check permissions and that no file occupies the parent path", e);
  }
  throw e;
}

When it happens

Trigger: fs.create("/var/data/out/file") where /var/data/out cannot be created because the running user lacks write permission on /var/data, or because a regular file named 'out' already occupies the path, or on unusual mounts (read-only NFS, full disk) where mkdir fails.

Common situations: Services running as a non-root user writing into system directories, containers with read-only volumes mounted at the target, a prior step that wrote a file where a directory tree is now expected, or disk/quota exhaustion on the local volume.

Related errors

• Parent directory doesn't exist: {}
• Mkdirs failed to create {} (exists={}, cwd={})
• File already exists: " + f
• mkdirs path arg is null
• Parent path is not a directory: " + parent

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1673d78f67c6f32a. Report an issue: GitHub.