apache/hadoop · error · PathIsNotDirectoryException

Is not a directory

Error message

Is not a directory

What it means

Thrown by CommandWithDestination.processArguments() when MULTIPLE source arguments are given and the destination exists but is a regular file. Multiple sources can only land inside a directory, so PathIsNotDirectoryException ('dst: Is not a directory') aborts before any transfer starts.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java:233

          dst = items[0];
          break;
        default:
          throw new PathIOException(pathString, "Too many matches");
      }
    }
  }

  @Override
  protected void processArguments(LinkedList<PathData> args)
  throws IOException {
    // if more than one arg, the destination must be a directory
    // if one arg, the dst must not exist or must be a directory
    if (args.size() > 1) {
      if (!dst.exists) {
        throw new PathNotFoundException(dst.toString());
      }
      if (!dst.stat.isDirectory()) {
        throw new PathIsNotDirectoryException(dst.toString());
      }
    } else if (dst.exists) {
      if (!dst.stat.isDirectory() && !overwrite) {
        LOG.debug("Destination file exists: {}", dst.stat);
        throw new PathExistsException(dst.toString());
      }
    } else if (!dst.parentExists()) {
      throw new PathNotFoundException(dst.toString())
          .withFullyQualifiedPath(dst.path.toUri().toString());
    }
    super.processArguments(args);
  }

  @Override
  protected void processPathArgument(PathData src)
  throws IOException {
    if (src.stat.isDirectory() && src.fs.equals(dst.fs)) {
      PathData target = getTargetPath(src);

View on GitHub (pinned to 2add963021)

Solutions

1. Make the destination a directory: remove/rename the file, 'hdfs dfs -mkdir' the target, and rerun
2. If you want a single merged file, concatenate explicitly ('hdfs dfs -getmerge' or '-concat') instead of multi-source copy
3. Copy to a directory path with a trailing separator to state intent clearly

Example fix

# before
hdfs dfs -put a.txt b.txt /user/me/out.bin
# after
hdfs dfs -rm /user/me/out.bin && hdfs dfs -mkdir /user/me/out.bin && hdfs dfs -put a.txt b.txt /user/me/out.bin

// before copying multiple sources, ensure destination is a directory, not a file
if (fs.exists(dst) && !fs.getFileStatus(dst).isDirectory()) {
  throw new IOException("multi-source copy needs a directory destination, but " + dst + " is a file");
}

static boolean canAcceptMultipleSources(FileSystem fs, Path dst) throws IOException {
  return !fs.exists(dst) ? false : fs.getFileStatus(dst).isDirectory();
}

try {
  shellRun("-put", a, b, dst);
} catch (PathIsNotDirectoryException e) {
  // e.getPath() is the file-shaped destination; move it aside and make a real directory
  shellRun("-mv", dst, dst + ".bak");
  shellRun("-mkdir", dst);
  shellRun("-put", a, b, dst);
}

When it happens

Trigger: 'hdfs dfs -put a.txt b.txt /user/me/single.txt' where single.txt is an existing file; copying several files onto an existing file target with -cp; destination created earlier as a file by a first single-file copy.

Common situations: A first run created a file at the destination and later runs add more sources; scripts unconditionally appending sources; confusing Unix 'cat'-style merge semantics with copy semantics.

Related errors

• No such file or directory
• Too many matches
• File exists
• Operation not supported
• Is a directory

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f42558718d1039fb. Report an issue: GitHub.