What causes "Hadoop home directory {} is not a directory." in apache/hadoop?

Inspect the entry: ls -ld "$HADOOP_HOME" — the mode must start with 'd'; on Windows use dir %HADOOP_HOME% and check for

apache/hadoop · error · FileNotFoundException

Hadoop home directory {} is not a directory.

Error message

Hadoop home directory {} is not a directory.

What it means

Shell.checkHadoopHome() validates the resolved Hadoop home directory (HADOOP_HOME env var or hadoop.home.dir system property). The path exists but java.io.File.isDirectory() is false — it is a regular file — so it cannot contain bin/ and Shell throws FileNotFoundException. The installation reference is shape-wrong, not merely missing.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:548

    // after stripping any quotes, check for home dir being non-empty
    if (home.isEmpty()) {
      throw new FileNotFoundException(E_HADOOP_PROPS_EMPTY);
    }

    // check that the hadoop home dir value
    // is an absolute reference to a directory
    File homedir = new File(home);
    if (!homedir.isAbsolute()) {
      throw new FileNotFoundException("Hadoop home directory " + homedir
          + " " + E_IS_RELATIVE);
    }
    if (!homedir.exists()) {
      throw new FileNotFoundException("Hadoop home directory " + homedir
          + " " + E_DOES_NOT_EXIST);
    }
    if (!homedir.isDirectory()) {
      throw new FileNotFoundException("Hadoop home directory " + homedir
          + " "+ E_NOT_DIRECTORY);
    }
    return homedir;
  }

  /**
   * The Hadoop home directory.
   */
  private static final File HADOOP_HOME_FILE;

  /**
   * Rethrowable cause for the failure to determine the hadoop
   * home directory
   */
  private static final IOException HADOOP_HOME_DIR_FAILURE_CAUSE;

  static {
    File home;

View on GitHub (pinned to 2add963021)

Solutions

Inspect the entry: ls -ld "$HADOOP_HOME" — the mode must start with 'd'; on Windows use dir %HADOOP_HOME% and check for <DIR>
If it is a file, extract the distribution and set HADOOP_HOME to the resulting directory (containing bin/)
Fix the hadoop.home.dir system property the same way if it is the source of the value
Replace any symlink at that path with a real directory or a link that resolves to one

Example fix

// before
export HADOOP_HOME=/opt/hadoop-3.3.6.tar.gz   # the archive, a file

// after
tar -xzf /opt/hadoop-3.3.6.tar.gz -C /opt/
export HADOOP_HOME=/opt/hadoop-3.3.6            # the extracted directory

Defensive patterns

Strategy: validation

Validate before calling

File home = new File(System.getenv().getOrDefault("HADOOP_HOME",
    System.getProperty("hadoop.home.dir", "")));
if (!home.isDirectory()) { // exists AND is a directory
  throw new IllegalStateException("HADOOP_HOME must be a directory, got: " + home);
}

Try / catch

try { Shell.getHadoopHome(); } catch (FileNotFoundException e) { log.error("Fix HADOOP_HOME: not a directory: {}", e.getMessage()); throw e; }

Prevention

Provision HADOOP_HOME exclusively from an extracted archive so it is a directory by construction
Add ls -ld $HADOOP_HOME to environment bootstrap scripts to assert the 'd' mode
In Docker, COPY the extracted tree (not the tarball) to the ENV-declared path

When it happens

Trigger: HADOOP_HOME or hadoop.home.dir names a file, e.g. the hadoop-3.3.6.tar.gz archive itself, a metadata file, or a symlink chain that ends at a file; then any Shell-dependent API (Windows file ops, ShellCommandExecutor, winutils lookup) triggers the check.

Common situations: Pointing HADOOP_HOME at the downloaded tarball instead of the extracted directory; a Docker COPY that placed a file at the path the env var names; symlink targets replaced by files during upgrades.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7a98444943ee727f. Report an issue: GitHub.