apache/hadoop · error · DiskErrorException

Cannot create directory: ${dir}

Error message

Cannot create directory: ${dir}

What it means

DiskChecker.checkDirInternal calls mkdirsWithExistsCheck() (a race-tolerant mkdir) and throws DiskErrorException('Cannot create directory: <dir>') when mkdir fails and the path still does not exist. This check guards every local storage directory DataNodes and NodeManagers use (data dirs, local dirs, log dirs), at startup and during periodic health checks.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DiskChecker.java:97

  /**
   * Create the directory if it doesn't exist and check that dir is
   * readable, writable and executable. Perform some disk IO to
   * ensure that the disk is usable for writes.
   *
   * @param dir dir.
   * @throws DiskErrorException disk problem.
   */
  public static void checkDirWithDiskIo(File dir)
      throws DiskErrorException {
    checkDirInternal(dir);
    doDiskIo(dir);
  }

  private static void checkDirInternal(File dir)
      throws DiskErrorException {    
    if (!mkdirsWithExistsCheck(dir)) {
      throw new DiskErrorException("Cannot create directory: "
                                   + dir.toString());
    }
    checkAccessByFileMethods(dir);
  }

  /**
   * Create the local directory if necessary, check permissions and also ensure
   * it can be read from and written into.
   *
   * @param localFS local filesystem
   * @param dir directory
   * @param expected permission
   * @throws DiskErrorException disk problem.
   * @throws IOException raised on errors performing I/O.
   */
  public static void checkDir(LocalFileSystem localFS, Path dir,
                              FsPermission expected)
      throws DiskErrorException, IOException {

View on GitHub (pinned to 2add963021)

Solutions

1. Create the directory manually as the daemon user (sudo -u hdfs mkdir -p <dir>) to surface the real OS error
2. chown the directory to the daemon user or chmod it writable
3. Check the filesystem is mounted read-write and the mount survives reboots
4. Correct the typo'd path in the configuration

Example fix

# before
sudo mkdir -p /data/dfs/dn && chmod 700 /data/dfs/dn   # owned by root, DataNode cannot create subdirs
# after
sudo mkdir -p /data/dfs/dn && sudo chown -R hdfs:hadoop /data/dfs/dn && sudo -u hdfs mkdir /data/dfs/dn/current

void preflightDirs(Path... dirs) throws IOException {
  for (Path p : dirs) {
    java.nio.file.Path d = java.nio.file.Paths.get(p.toString());
    java.nio.file.Files.createDirectories(d);          // throws with the real OS reason
    if (!java.nio.file.Files.isWritable(d)) {
      throw new IOException("not writable by this user: " + d);
    }
  }
}

try {
  DiskChecker.checkDir(localFS, dir, expected);
} catch (DiskErrorException e) {
  // surface immediately at startup with the failing dir in the message
  throw new RuntimeException("storage dir failed check: " + dir, e);
}

When it happens

Trigger: A parent directory not writable by the daemon user; a path component exists as a regular file; a read-only or unmounted filesystem returning EACCES/EROFS from mkdir.

Common situations: dfs.datanode.data.dir, yarn.nodemanager.local-dirs or yarn.nodemanager.log-dirs pointing at unwritable paths; directory ownership changing after cluster handover; mounts lost after reboot; SELinux denials.

Related errors

• Directory is not readable: ${dir}
• Directory is not writable: ${dir}
• Directory is not executable: ${dir}
• Error checking directory ${dir}
• Not a directory: ${dir}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6489adac106b2072. Report an issue: GitHub.