apache/hadoop · error · PathIsDirectoryException

Is a directory

Error message

Is a directory

What it means

Thrown by CommandWithDestination.processPath(src, dst) when the source is a directory and the command was not run recursively (isRecursive() false). Copying a directory non-recursively has no defined output, so PathIsDirectoryException aborts with 'src: Is a directory'. Note most modern copy commands (put, get, cp in current releases) set setRecursive(true) internally, so this typically surfaces from commands or subclasses that leave recursion off (e.g. getmerge sources, custom commands extending CommandWithDestination).

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java:295

  protected void processPath(PathData src) throws IOException {
    processPath(src, getTargetPath(src));
  }
  
  /**
   * Called with a source and target destination pair
   * @param src for the operation
   * @param dst for the operation
   * @throws IOException if anything goes wrong
   */
  protected void processPath(PathData src, PathData dst) throws IOException {
    if (src.stat.isSymlink()) {
      // TODO: remove when FileContext is supported, this needs to either
      // copy the symlink or deref the symlink
      throw new PathOperationException(src.toString());        
    } else if (src.stat.isFile()) {
      copyFileToTarget(src, dst);
    } else if (src.stat.isDirectory() && !isRecursive()) {
      throw new PathIsDirectoryException(src.toString());
    }
  }

  @Override
  protected void recursePath(PathData src) throws IOException {
    PathData savedDst = dst;
    try {
      // modify dst as we descend to append the basename of the
      // current directory being processed
      dst = getTargetPath(src);
      final boolean preserveRawXattrs =
          checkPathsForReservedRaw(src.path, dst.path);
      if (dst.exists) {
        if (!dst.stat.isDirectory()) {
          throw new PathIsNotDirectoryException(dst.toString());
        }
      } else {
        if (!dst.fs.mkdirs(dst.path)) {

View on GitHub (pinned to 2add963021)

Solutions

1. Use the recursive form of the command where available (e.g. 'hdfs dfs -cp -r' style flows) so directories recurse
2. Point the command at the files inside the directory instead of the directory itself
3. If writing a CommandWithDestination subclass, call setRecursive(true) in processOptions when directory sources are legal

Example fix

# before (command with recursion off, source is a dir)
hdfs dfs -cp /user/me/somedir /backup
# after
hdfs dfs -cp -r /user/me/somedir /backup  # current cp sets recursive internally; older/custom commands need it

// if recursion is not enabled, ensure every source is a regular file
for (Path s : sources) {
  FileStatus st = fs.getFileStatus(s);
  if (st.isDirectory()) {
    throw new IOException("source is a directory but recursion is off: " + s);
  }
}

static boolean allSourcesAreFiles(FileSystem fs, List<Path> srcs) throws IOException {
  for (Path p : srcs) if (fs.getFileStatus(p).isDirectory()) return false;
  return true;
}

try {
  shellRun(copyCmd, src, dst);
} catch (PathIsDirectoryException e) {
  // e.getPath() is the directory source; retry in recursive mode
  shellRun(copyCmd, "-r", e.getPath().toString(), dst);
}

When it happens

Trigger: A source argument that expands to a directory while isRecursive() is false; passing a directory to a command whose USAGE only accepts files; a glob matching a directory among file results.

Common situations: Assuming every -cp variant needs no -r (older documentation); custom FsShell plugins built on CommandWithDestination that forget setRecursive(true); directory accidentally matched by a loose glob.

Related errors

• No such file or directory
• Too many matches
• Is not a directory
• File exists
• Operation not supported

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/60103b556cecc42d. Report an issue: GitHub.