apache/hadoop · error · PathIsDirectoryException

Is a directory

Error message

Is a directory

What it means

PathIsDirectoryException ('Is a directory') thrown by Head.processPath (Head.java:66) when 'hadoop fs -head' is applied to a directory. Head opens the path with the sequential read policy and copies only endingOffset bytes to stdout; a directory cannot be opened as a stream, so it is rejected before openFile().

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Head.java:66

  private long endingOffset = 1024;

  @Override
  protected void processOptions(LinkedList<String> args) throws IOException {
    CommandFormat cf = new CommandFormat(1, 1);
    cf.parse(args);
  }

  @Override
  protected List<PathData> expandArgument(String arg) throws IOException {
    List<PathData> items = new LinkedList<PathData>();
    items.add(new PathData(arg, getConf()));
    return items;
  }

  @Override
  protected void processPath(PathData item) throws IOException {
    if (item.stat.isDirectory()) {
      throw new PathIsDirectoryException(item.toString());
    }

    dumpToOffset(item);
  }

  private void dumpToOffset(PathData item) throws IOException {
    try (FSDataInputStream in = item.openFile(
        FS_OPTION_OPENFILE_READ_POLICY_SEQUENTIAL)) {
      IOUtils.copyBytes(in, System.out, endingOffset, false);
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Point at a file inside: 'hadoop fs -head /data/somedir/part-00000'
2. List first ('hadoop fs -ls /data/somedir') to pick the file to head
3. For structured data checks prefer 'hadoop fs -cat <file> | head -n 10' on a concrete part file
4. In code, gate on stat.isFile() before calling openFile()

Example fix

# before
hadoop fs -head /events/dt=2026-08-21    # Is a directory

# after
hadoop fs -head /events/dt=2026-08-21/part-00000

if (item.stat.isDirectory()) {
  throw new PathIsDirectoryException(item.toString());
}

static boolean isHeadableFile(FileStatus st) {
  return st != null && st.isFile();
}

try {
  try (FSDataInputStream in = item.openFile(
      FS_OPTION_OPENFILE_READ_POLICY_SEQUENTIAL)) {
    IOUtils.copyBytes(in, System.out, endingOffset, false);
  }
} catch (PathIsDirectoryException e) {
  // list the directory and head a specific part file
}

When it happens

Trigger: 'hadoop fs -head /data/somedir'; peeking at a partition root; a script sanity-checking '$FILE' that actually points at a directory.

Common situations: Quick-peek workflows on generated data where users head the output dir instead of a part file; typo in the path dropping the file component; onboarding examples copied from cat/grep habits.

Related errors

• Is a directory
• Is a directory
• Is a directory
• Illegal option {}
• Not enough arguments: expected {} but got {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/98b01cf00b14ed0f. Report an issue: GitHub.