apache/hadoop · error · FileNotFoundException

File {} does not exist.

Error message

File {} does not exist.

What it means

Helper behind GET op=LISTSTATUS. ClientProtocol.getListing returns null when the path is not an existing directory, and getDirectoryListing turns that into FileNotFoundException('File <p> does not exist.') -> HTTP 404. Per the code comment, the first listing page is fetched before the HTTP 200 body starts streaming, precisely so this failure does not get stuck mid-response; the same helper also serves pagination (startAfter), so a directory deleted mid-listing fails on the next page.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java:1694

    int start = 0;
    if ((path.length() == start) || // empty path
        (lastSlash == start && path.length() == start + 1)) { // at root
      return null;
    }
    String parent;
    if (lastSlash == -1) {
      parent = org.apache.hadoop.fs.Path.CUR_DIR;
    } else {
      parent = path.substring(0, lastSlash == start ? start + 1 : lastSlash);
    }
    return parent;
  }

  private static DirectoryListing getDirectoryListing(final ClientProtocol cp,
      final String p, byte[] startAfter) throws IOException {
    final DirectoryListing listing = cp.getListing(p, startAfter, false);
    if (listing == null) { // the directory does not exist
      throw new FileNotFoundException("File " + p + " does not exist.");
    }
    return listing;
  }
  
  private static StreamingOutput getListingStream(final ClientProtocol cp,
      final String p) throws IOException {
    // allows exceptions like FNF or ACE to prevent http response of 200 for
    // a failure since we can't (currently) return error responses in the
    // middle of a streaming operation
    final DirectoryListing firstDirList = getDirectoryListing(cp, p,
        HdfsFileStatus.EMPTY_NAME);

    // must save ugi because the streaming object will be executed outside
    // the remote user's ugi
    final UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
    return new StreamingOutput() {
      @Override
      public void write(final OutputStream outstream) throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

1. Confirm the path exists and is a directory first (op=GETFILESTATUS then check isDirectory)
2. If listing a file was intended, use op=GETFILESTATUS — LISTSTATUS is for directories only
3. On 404 mid-pagination, stop and re-list from the parent; the directory changed underneath you
4. Encode the path exactly (spaces, %, #) before sending

Example fix

# before
curl "http://nn:9870/webhdfs/v1/data/old_dir?op=LISTSTATUS"   # 404 File /data/old_dir does not exist.
curl "http://nn:9870/webhdfs/v1/data/file.txt?op=LISTSTATUS" # 404 too: files cannot be LISTSTATUSed

# after
curl "http://nn:9870/webhdfs/v1/data/file.txt?op=GETFILESTATUS"  # file -> use status, not listing
curl "http://nn:9870/webhdfs/v1/data/new_dir?op=LISTSTATUS"      # directory -> 200

FileStatus st = webhdfs.getFileStatus(dir);
if (!st.isDirectory()) {
  throw new IllegalArgumentException(dir + " is a file; LISTSTATUS requires a directory");
}

try {
  listing = getDirectoryListing(cp, dir, HdfsFileStatus.EMPTY_NAME);
} catch (FileNotFoundException e) { // 404: dir deleted, or path is a file
  return ListingResult.absent(dir);
}

When it happens

Trigger: op=LISTSTATUS on a deleted or never-existing directory; on a plain file (getListing only lists directories, so a regular file also yields this misleading 'does not exist' message); pagination continuing after the directory was removed between pages.

Common situations: Directory cleanup racing a UI/job listing; LISTSTATUS issued where GETFILESTATUS was meant (file vs directory); tools assuming a fixed directory exists after a re-layout.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• File does not exist: {}
• Storage policy name is empty.
• {} is not supported
• {} is not supported
• {} parameter is not null.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/14e56449b7bea974. Report an issue: GitHub.