apache/hadoop · error · IOException

Path %s is a directory.

Error message

Path %s is a directory.

What it means

Thrown by SFTPFileSystem.open(Path f, int bufferSize) when getFileStatus on the resolved path reports a directory. Opening a read stream against a directory is not meaningful over SFTP, so the adapter disconnects the channel and throws IOException(E_PATH_DIR).

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java:520

  @Override
  public URI getUri() {
    return uri;
  }

  @Override
  public FSDataInputStream open(Path f, int bufferSize) throws IOException {
    ChannelSftp channel = connect();
    Path workDir;
    try {
      workDir = new Path(channel.pwd());
    } catch (SftpException e) {
      throw new IOException(e);
    }
    Path absolute = makeAbsolute(workDir, f);
    FileStatus fileStat = getFileStatus(channel, absolute);
    if (fileStat.isDirectory()) {
      disconnect(channel);
      throw new IOException(String.format(E_PATH_DIR, f));
    }
    try {
      // the path could be a symbolic link, so get the real path
      absolute = new Path("/", channel.realpath(absolute.toUri().getPath()));
    } catch (SftpException e) {
      throw new IOException(e);
    }
    return new FSDataInputStream(
        new SFTPInputStream(channel, absolute, statistics)){
      @Override
      public void close() throws IOException {
        try {
          super.close();
        } finally {
          disconnect(channel);
        }
      }
    };

View on GitHub (pinned to 2add963021)

Solutions

  1. Guard the open with a file check: if (!fs.getFileStatus(f).isFile()) fail fast with your own message.
  2. If the intent was to enumerate contents, use fs.listStatus(dir) instead of open().
  3. Validate configuration-supplied input paths at application startup (isFile) before any stream is opened.

Example fix

// before
FSDataInputStream in = fs.open(inputPath); // throws if inputPath is a directory

// after
FileStatus st = fs.getFileStatus(inputPath);
if (st.isDirectory()) {
  throw new IllegalArgumentException("input must be a file: " + inputPath);
}
FSDataInputStream in = fs.open(inputPath);
Defensive patterns

Strategy: validation

Validate before calling

FileStatus st = fs.getFileStatus(f);
if (st.isDirectory()) {
  throw new IllegalArgumentException("expected a file, got directory: " + f);
}
FSDataInputStream in = fs.open(f);

Type guard

private static boolean isRegularFile(FileSystem fs, Path p) throws IOException {
  FileStatus st = fs.getFileStatus(p);
  return !st.isDirectory();
}

Try / catch

try {
  in = fs.open(f);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().endsWith("is a directory.")) {
    // input path points at a directory -> fix configuration
  } else {
    throw e;
  }
}

Prevention

When it happens

Trigger: Calling fs.open(f) (directly or via open(f, bufferSize)) on an sftp:// path whose lstat attributes have the directory bit set; typically a variable, glob, or config property that points at a directory rather than a file.

Common situations: A reader configured with a job input directory instead of a specific file; a path variable that resolved to the parent directory; code that assumed open() on a directory would fail differently (e.g., FileNotFoundException) or return an empty stream.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b5233e8bdd1b82ed. Report an issue: GitHub.