What causes "File does not exist: {}" in apache/hadoop?

Verify with `hdfs dfs -ls` against the same fs.defaultFS the code uses

apache/hadoop · error · FileNotFoundException

File does not exist: {}

Error message

File does not exist: {}

What it means

Hdfs.getFileStatus asks DFSClient.getFileInfo for the path; a null response means the NameNode has no entry for it, which becomes FileNotFoundException("File does not exist: " + path). This is the FileContext-level twin of the FileSystem version — thrown for a genuinely missing path, not for permission problems (those surface as AccessControlException) and not for broken symlinks (UnresolvedLinkException).

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/Hdfs.java:153

    return dfs.getFileChecksumWithCombineMode(getUriPath(f), Long.MAX_VALUE);
  }

  /**
   * {@inheritDoc}
   *
   * If the given path is a symlink, the path will be resolved to a target path
   * and it will get the resolved path's FileStatus object. It will not be
   * represented as a symlink and isDirectory API returns true if the resolved
   * path is a directory, false otherwise.
   */
  @Override
  public FileStatus getFileStatus(Path f) 
      throws IOException, UnresolvedLinkException {
    HdfsFileStatus fi = dfs.getFileInfo(getUriPath(f));
    if (fi != null) {
      return fi.makeQualified(getUri(), f);
    } else {
      throw new FileNotFoundException("File does not exist: " + f.toString());
    }
  }

  /**
   * Synchronize client metadata state with Active NameNode.
   * <p>
   * In HA the client synchronizes its state with the Active NameNode
   * in order to guarantee subsequent read consistency from Observer Nodes.
   * @throws IOException
   */
  @Override
  public void msync() throws IOException {
    dfs.msync();
  }

  @Override
  public FileStatus getFileLinkStatus(Path f) 
      throws IOException, UnresolvedLinkException {

View on GitHub (pinned to 2add963021)

Solutions

Verify with `hdfs dfs -ls` against the same fs.defaultFS the code uses
Catch FileNotFoundException and treat it as an expected outcome (skip, recreate, or report)
For racy paths, re-check with fc.util().exists() or listStatus and tolerate absence

Example fix

// before
FileStatus st = fc.getFileStatus(p);

// after
FileStatus st;
try {
  st = fc.getFileStatus(p);
} catch (FileNotFoundException e) {
  return; // treat as absent
}

Defensive patterns

Strategy: try-catch

Validate before calling

if (!fc.util().exists(p)) {
  // absent: skip, create parent, or report — do not call getFileStatus
}

Try / catch

try {
  FileStatus st = fc.getFileStatus(p);
} catch (FileNotFoundException e) {
  // expected when racing with deletions; handle absence explicitly
}

Prevention

Use fc.util().exists() when absence is normal and races are unlikely
Resolve paths against a known fs.defaultFS to avoid stat-ing the wrong cluster
Treat FileNotFoundException as control flow, not an error to log at FATAL

When it happens

Trigger: fc.getFileStatus(path) on a path that was deleted, never existed, contains a typo, or is being addressed through the wrong authority (a different cluster or nameservice) where it legitimately does not exist.

Common situations: TOCTOU races where a file is deleted between listing and stat; relative paths resolved against an unexpected working directory/defaultFS; scripts pointed at the standby or DR cluster after failover.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8d8cad9779d2e4ae. Report an issue: GitHub.