What causes "create(): Mkdirs failed to create: {parent}" in apache/hadoop?

Create the parent explicitly first: if (!fs.mkdirs(parent)) { ... } and inspect the FTP server reply for the failing MKD

apache/hadoop · error · IOException

create(): Mkdirs failed to create: {parent}

Error message

create(): Mkdirs failed to create: {parent}

What it means

Before storing a file, create() ensures the parent directory chain exists by calling mkdirs; if that returns false (some MKD was refused), create aborts with this IOException after disconnecting. When the target's parent is null it is normalized to '/'.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FTPFileSystem.java:338

    try {
      status = getFileStatus(client, file);
    } catch (FileNotFoundException fnfe) {
      status = null;
    }
    if (status != null) {
      if (overwrite && !status.isDirectory()) {
        delete(client, file, false);
      } else {
        disconnect(client);
        throw new FileAlreadyExistsException("File already exists: " + file);
      }
    }
    
    Path parent = absolute.getParent();
    if (parent == null || !mkdirs(client, parent, FsPermission.getDirDefault())) {
      parent = (parent == null) ? new Path("/") : parent;
      disconnect(client);
      throw new IOException("create(): Mkdirs failed to create: " + parent);
    }
    client.allocate(bufferSize);
    // Change to parent directory on the server. Only then can we write to the
    // file on the server by opening up an OutputStream. As a side effect the
    // working directory on the server is changed to the parent directory of the
    // file. The FTP client connection is closed when close() is called on the
    // FSDataOutputStream.
    client.changeWorkingDirectory(parent.toUri().getPath());
    OutputStream outputStream = client.storeFileStream(file.getName());

    if (!FTPReply.isPositivePreliminary(client.getReplyCode())) {
      // The ftpClient is an inconsistent state. Must close the stream
      // which in turn will logout and disconnect from FTP server
      if (outputStream != null) {
        IOUtils.closeStream(outputStream);
      }
      disconnect(client);
      throw new IOException("Unable to create file: " + file + ", Aborting");

View on GitHub (pinned to 2add963021)

Solutions

Create the parent explicitly first: if (!fs.mkdirs(parent)) { ... } and inspect the FTP server reply for the failing MKD
Verify write permission on the intended parent via getFileStatus(parent).getPermission() containing FsAction.WRITE
Make sure no intermediate component is a file — mkdirs cannot traverse a file
Check server disk space / quota

Example fix

// before
FSDataOutputStream out = fs.create(new Path("/upload/alice/out/part-0"), true);
// IOException: create(): Mkdirs failed to create: /upload/alice/out

// after
Path parent = new Path("/upload/alice/out");
if (!fs.mkdirs(parent)) {
  throw new IOException("Cannot create parent " + parent + " - check write permission");
}
FSDataOutputStream out = fs.create(new Path(parent, "part-0"), true);

Defensive patterns

Strategy: validation

Validate before calling

Path parent = file.getParent() != null ? file.getParent() : new Path("/");
if (!fs.exists(parent) && !fs.mkdirs(parent)) {
  throw new IOException("Cannot create parent directory " + parent
      + " - check FTP write permission / disk space");
}
FSDataOutputStream out = fs.create(file, true);

Try / catch

try {
  out = fs.create(file, true);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("Mkdirs failed")) {
    throw new IOException("FTP server refused to create parent dirs for " + file
        + " - verify write permission on ancestor dirs", e);
  }
  throw e;
}

Prevention

Pre-create directory trees during deployment where permissions are known-good
Ensure no intermediate path component is a file before writing nested outputs
Grant the FTP account write access on the upload root once, not per-job

When it happens

Trigger: FTP user lacks write permission on the deepest existing ancestor directory; an intermediate path component already exists as a file so MKD fails mid-chain; server quota exhausted or read-only share.

Common situations: Writing into /upload/<user>/ trees where only /upload is writable; path components created by others with restrictive modes; disk full on the FTP server.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7aa93562e5641715. Report an issue: GitHub.