apache/hadoop · error · PathNotFoundException

No such file or directory

Error message

No such file or directory

What it means

Thrown by the shell 'mkdir' command (Mkdir.processNonexistentPath, Mkdir.java:84) when the directory to create does not exist AND its parent directory also does not exist. This check only runs when the -p flag was NOT given; PathNotFoundException's default message is 'No such file or directory'. It exists because plain mkdir, unlike mkdirs(), refuses to silently create intermediate directories.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Mkdir.java:84

    }
  }

  @Override
  protected void processNonexistentPath(PathData item) throws IOException {
    if (!createParents) {
      // check if parent exists. this is complicated because getParent(a/b/c/) returns a/b/c, but
      // we want a/b
      final Path itemPath = new Path(item.path.toString());
      final Path itemParentPath = itemPath.getParent();

      if(itemParentPath == null) {
        throw new PathNotFoundException(String.format(
            "Item: %s parent's path is null. This can happen if mkdir is " +
                "called on root, so there's no parent.", itemPath.toString()));
      }

      if (!item.fs.exists(itemParentPath)) {
        throw new PathNotFoundException(itemParentPath.toString());
      }
    }
    if (!item.fs.mkdirs(item.path)) {
      throw new PathIOException(item.toString());
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Re-run with the -p flag: 'hadoop fs -mkdir -p /user/alice/newdir' (createParents=true skips the parent check)
2. Create the missing parent level(s) explicitly first: 'hadoop fs -mkdir /user/alice' then 'hadoop fs -mkdir /user/alice/newdir'
3. Check for typos in the intermediate path components with 'hadoop fs -ls' on the expected parent
4. In code, call FileSystem.mkdirs(path) instead of issuing a shell mkdir without -p

Example fix

# before
hadoop fs -mkdir /data/logs/2026/08
# mkdir: `/data/logs/2026/08': No such file or directory

# after
hadoop fs -mkdir -p /data/logs/2026/08

// before creating, ensure the parent chain exists
Path dst = new Path("/user/alice/newdir");
if (!fs.exists(dst.getParent())) {
  fs.mkdirs(dst.getParent()); // creates all intermediates
}
fs.mkdirs(dst);

catch (PathNotFoundException e) { /* create parents, then retry once */ fs.mkdirs(parent); fs.mkdirs(dst); }

When it happens

Trigger: Running 'hadoop fs -mkdir /user/alice/newdir' when /user/alice does not exist. Any shell mkdir invocation (without -p) whose target's parent is missing, including parents missing due to a typo in an intermediate path segment. Only reached via processNonexistentPath, i.e. the target itself does not already exist.

Common situations: Scripts ported from Linux 'mkdir -p' habits that assume parents are auto-created; bootstrapping a new user/tool directory tree on a fresh HDFS namespace; typo'd intermediate path components; CI jobs that assume a previous step created the base directory.

Related errors

• Input/output error
• No such file or directory
• Item: %s parent's path is null. This can happen if mkdir is
• Could not set replication for:
• Cannot create directory {rootPath}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c1d0b10e8fd8762e. Report an issue: GitHub.