apache/hadoop · error · FileNotFoundException

Can't open {path} because it is a directory

Error message

Can't open {path} because it is a directory

What it means

Thrown by AliyunOSSFileSystem.open(): getFileStatus(path) succeeded but reported a directory, and OSS has no way to read a 'directory' as a byte stream (directories are just key prefixes, possibly marker objects). open() therefore fails with FileNotFoundException("Can't open " + path + " because it is a directory").

Source

Thrown at hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java:609

        FileStatus fileStatus = getFileStatus(fPart);
        if (fileStatus.isDirectory()) {
          // If path exists and a directory, exit
          break;
        } else {
          throw new FileAlreadyExistsException(String.format(
              "Can't make directory for path '%s', it is a file.", fPart));
        }
      } catch (FileNotFoundException fnfe) {
      }
      fPart = fPart.getParent();
    } while (fPart != null);
  }

  @Override
  public FSDataInputStream open(Path path, int bufferSize) throws IOException {
    final FileStatus fileStatus = getFileStatus(path);
    if (fileStatus.isDirectory()) {
      throw new FileNotFoundException("Can't open " + path +
          " because it is a directory");
    }

    return new FSDataInputStream(new AliyunOSSInputStream(getConf(),
        new SemaphoredDelegatingExecutor(
            boundedThreadPool, maxReadAheadPartNumber, true),
        maxReadAheadPartNumber, store, pathToKey(path), fileStatus.getLen(),
        statistics));
  }

  @Override
  public boolean rename(Path srcPath, Path dstPath) throws IOException {
    if (srcPath.isRoot()) {
      // Cannot rename root of file system
      if (LOG.isDebugEnabled()) {
        LOG.debug("Cannot rename the root of a filesystem");
      }
      return false;

View on GitHub (pinned to 2add963021)

Solutions

1. Open a concrete file: list the directory with fs.listStatus(dir) and open each file, or construct the exact part file path
2. Use a directory-aware reader (e.g., Spark/Hive reading the partition path) instead of FileSystem.open
3. If the path should be a file, check whether a write failed or the key was created as a marker by another tool, and rewrite it

Example fix

// before
FSDataInputStream in = fs.open(new Path("oss://bucket/out")); // out/ is a dir

// after
for (FileStatus st : fs.listStatus(new Path("oss://bucket/out"))) {
  if (st.isFile()) { try (FSDataInputStream in = fs.open(st.getPath())) { /* read */ } }
}

FileStatus st = fs.getFileStatus(path);
if (st.isDirectory()) {
  throw new IOException("open() target is a directory: " + path);
}

private boolean isReadableFile(FileSystem fs, Path p) throws IOException {
  FileStatus st = fs.getFileStatus(p);
  return st.isFile();
}

catch (FileNotFoundException e) { if (e.getMessage().contains("because it is a directory")) { /* list and read children instead */ } else throw e; }

When it happens

Trigger: Calling fs.open(path) / fs.openFile(path) where path resolves to a directory marker or a prefix containing children; reading the output of a mapreduce job at the output directory instead of a specific part file; passing a partition directory to a reader expecting a file.

Common situations: Users opening the job output directory instead of part-00000; passing oss://bucket/dataset/date=2024-01-01 (a partition dir) to a tool that expects one file; marker objects left by other writers making a nonexistent-file path stat as directory.

Related errors

• Credentials should not be null.
• Invalid credentials
• Stream closed.
• Failed to multipart upload to oss, abort it.
• Multi-part upload with id '{uploadId}' to {key}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7219afd61b25cee3. Report an issue: GitHub.