apache/hadoop · error · FileNotFoundException

File " + f + " does not exist

Error message

File " + f + " does not exist

What it means

RawLocalFileSystem.listStatus(Path) stats the local file first and throws FileNotFoundException('File ... does not exist') when the path is absent. This mirrors the contract of the abstract FileSystem: listing a missing path is an error, not an empty array. The underlying cause is a plain missing local file or symlink chain.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:817

        (FileUtil.listFiles(f).length != 0)) {
      throw new IOException("Directory " + f.toString() + " is not empty");
    }
    return FileUtil.fullyDelete(f);
  }
 
  /**
   * {@inheritDoc}
   *
   * (<b>Note</b>: Returned list is not sorted in any given order,
   * due to reliance on Java's {@link File#list()} API.)
   */
  @Override
  public FileStatus[] listStatus(Path f) throws IOException {
    File localf = pathToFile(f);
    FileStatus[] results;

    if (!localf.exists()) {
      throw new FileNotFoundException("File " + f + " does not exist");
    }

    if (localf.isDirectory()) {
      String[] names = FileUtil.list(localf);
      results = new FileStatus[names.length];
      int j = 0;
      for (int i = 0; i < names.length; i++) {
        try {
          // Assemble the path using the Path 3 arg constructor to make sure
          // paths with colon are properly resolved on Linux
          results[j] = getFileStatus(new Path(f, new Path(null, null,
                                                          names[i])));
          j++;
        } catch (FileNotFoundException e) {
          // ignore the files not found since the dir list may have have
          // changed since the names[] list was generated.
        }
      }

View on GitHub (pinned to 2add963021)

Solutions

1. Verify on disk: ls -ld the exact path; fix typos or update the configuration to the real location.
2. Guard in code: if (!fs.exists(p)) handle-missing (create, skip, or fail with your own message).
3. Order pipeline stages so inputs exist before listing; poll for a _SUCCESS marker before consuming.
4. For symlinks, confirm the target exists with Files.readSymbolicLink / readlink.

Example fix

// before
for (FileStatus st : fs.listStatus(inputDir)) { ... } // throws if missing

// after
if (!fs.exists(inputDir)) {
  throw new FileNotFoundException("Input not ready: " + inputDir);
}
for (FileStatus st : fs.listStatus(inputDir)) { ... }

if (!fs.exists(dir)) {
  throw new FileNotFoundException(
      "Input directory missing (producer not run?): " + dir);
}
FileStatus[] entries = fs.listStatus(dir);

try {
  return fs.listStatus(dir);
} catch (FileNotFoundException e) {
  // distinguish expected-missing from misconfiguration before swallowing
  LOG.warn("Input {} missing; treating as empty", dir);
  return new FileStatus[0];
}

When it happens

Trigger: Calling fs.listStatus(p) where p does not exist on local disk: input directory moved or renamed, typo in configuration, a job submitted before the input stage produced the directory, or a dangling symlink (stat on the link target fails).

Common situations: Pipeline stages run out of order so the consumer lists before the producer writes, input paths from core-site.xml pointing at renamed mounts, containers where the volume was not mounted, or code ported from an environment where the directory existed.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• File " + f + " not found
• File " + p + " does not exist
• Parent directory doesn't exist: {}
• File {} not found in {}
• key + ": No such file or directory."

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8e74982230a65015. Report an issue: GitHub.