apache/hadoop · error · FileNotFoundException

Can't open {} because it is a directory

Error message

Can't open {} because it is a directory

What it means

OBSFileSystem.open(Path, int) resolves the path's FileStatus and refuses to open directories: if isDirectory() is true it throws FileNotFoundException('Can't open <path> because it is a directory'). In object stores a 'directory' is a marker (zero-byte key ending in '/') or an implied prefix, which has no readable content, so the connector fails fast with the standard 'not a file' signal rather than returning an empty stream. This matches the contract of FileSystem.open for other object-store implementations.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSFileSystem.java:659

  protected URI canonicalizeUri(final URI rawUri) {
    return OBSLoginHelper.canonicalizeUri(rawUri, getDefaultPort());
  }

  /**
   * Open an FSDataInputStream at the indicated Path.
   *
   * @param f          the file path to open
   * @param bufferSize the size of the buffer to be used
   * @return the FSDataInputStream for the file
   * @throws IOException on any failure to open the file
   */
  @Override
  public FSDataInputStream open(final Path f, final int bufferSize)
      throws IOException {
    LOG.debug("Opening '{}' for reading.", f);
    final FileStatus fileStatus = getFileStatus(f);
    if (fileStatus.isDirectory()) {
      throw new FileNotFoundException(
          "Can't open " + f + " because it is a directory");
    }

    return new FSDataInputStream(
        new OBSInputStream(bucket, OBSCommonUtils.pathToKey(this, f),
            fileStatus.getLen(),
            obs, statistics, readAheadRange, this));
  }

  /**
   * Create an FSDataOutputStream at the indicated Path with write-progress
   * reporting.
   *
   * @param f           the file path to create
   * @param permission  the permission to set
   * @param overwrite   if a file with this name already exists, then if true,
   *                    the file will be overwritten, and if false an error will
   *                    be thrown

View on GitHub (pinned to 2add963021)

Solutions

1. Check getFileStatus(f).isFile() (or use fs.exists plus isFile) before calling open
2. Fix the upstream path construction — usually the code built a directory path where the file key was intended (trailing slash, wrong join)
3. If the path may legitimately be either, branch on isDirectory() and read only in the file case
4. Delete the stray directory marker if it was created by mistake, then write/open the file key

Example fix

// before
FSDataInputStream in = fs.open(new Path("/data/out")); // if /data/out is a dir -> FileNotFoundException

// after
Path p = new Path("/data/out");
if (!fs.getFileStatus(p).isFile()) {
  throw new IllegalArgumentException("expected a file at " + p);
}
FSDataInputStream in = fs.open(p);

FileStatus st = fs.getFileStatus(f);
if (st.isDirectory()) {
  throw new IllegalArgumentException("cannot open a directory: " + f);
}
FSDataInputStream in = fs.open(f, bufferSize);

try {
  return fs.open(f, bufferSize);
} catch (FileNotFoundException e) {
  if (String.valueOf(e.getMessage()).contains("because it is a directory")) {
    // path bug: the target is a directory; fix path resolution upstream
  }
  // genuine missing file handling
  throw e;
}

When it happens

Trigger: Calling fs.open() on a path created with fs.mkdirs() or marked by a directory placeholder object; opening a path that a previous step created as a directory marker; glob/rename logic that resolves a directory where a file was expected.

Common situations: Pipelines where the same path is used as both output directory (MapReduce/Spark output committers create directories) and later opened as a file; racing writers where one creates the parent structure while another opens the leaf path; confusing 'directory exists' checks with 'file exists' checks before open.

Related errors

• {} is a directory
• Stream is closed!
• Cannot seek to a negative offset
• Attempted to seek or read past the end of the file
• Requested more bytes than destination buffer size: request l

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/08091a9d218a4cb8. Report an issue: GitHub.