apache/hadoop · error · FileNotFoundException

No such file or directory: %s

Error message

No such file or directory: %s

What it means

innerFileStatus() throws FileNotFoundException("No such file or directory: <qualifiedPath>") when getFileStatusOrNull returns null, i.e. no object or directory marker exists at the key. This is the primary 'path does not exist' signal of RawFileSystem and escapes through getFileStatus, open, delete, listStatus and rename source checks.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java:523

      throw new FileNotFoundException(e.getMessage());
    }
  }


  /**
   * Get the file status of given path.
   *
   * @param f the path
   * @return {@link RawFileStatus} describe file status info.
   * @throws FileNotFoundException       if the path doesn't exist.
   * @throws ParentNotDirectoryException if the path is locating under an existing file, which is
   *                                     not allowed in directory bucket case.
   */
  RawFileStatus innerFileStatus(Path f) throws ParentNotDirectoryException, FileNotFoundException {
    Path qualifiedPath = f.makeQualified(uri, workingDir);
    RawFileStatus fileStatus = getFileStatusOrNull(qualifiedPath);
    if (fileStatus == null) {
      throw new FileNotFoundException(
          String.format("No such file or directory: %s", qualifiedPath));
    }
    return fileStatus;
  }

  /**
   * The different with {@link RawFileSystem#getFileStatus(Path)} is that:
   * 1. throw  {@link ParentNotDirectoryException} if the path is locating under an existing file in
   * directory bucket case, but {@link RawFileSystem#getFileStatus(Path)} will ignore whether the
   * invalid path and throw {@link FileNotFoundException}
   * 2. return null if the path doesn't exist instead of throwing {@link FileNotFoundException}.
   *
   * @param path the object path.
   * @return null if the path doesn't exist.
   * @throws ParentNotDirectoryException if the path is locating under an existing file, which is
   *                                     not allowed in directory bucket case.
   */
  public RawFileStatus getFileStatusOrNull(final Path path) throws ParentNotDirectoryException {

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the full qualified path (makeQualified / fs.getWorkingDirectory) and bucket spelling
2. Check existence first: fs.exists(p) or fs.getFileStatusOrNull(p) (public on RawFileSystem) returning null
3. In recovery code, catch FileNotFoundException and retry/skip with backoff for eventually-produced files
4. List the parent directory to see what actually exists: fs.listStatus(p.getParent())

Example fix

// before
FSDataInputStream in = fs.open(new Path("out/part-0")); // FileNotFoundException if absent

// after
Path p = new Path("out/part-0");
if (fs.exists(p)) { try (FSDataInputStream in = fs.open(p)) { ... } }

if (fs.exists(p)) {
  try (FSDataInputStream in = fs.open(p)) { /* read */ }
}

try (FSDataInputStream in = fs.open(p)) { /* read */ }
catch (FileNotFoundException e) {
  // missing: log qualified path, verify bucket/prefix, skip or produce empty result
}

When it happens

Trigger: fs.open(p), fs.getFileStatus(p), fs.delete(p, ...) or rename with src=p on a tos:// path whose object key does not exist (and is not the root); paths whose working-directory qualification resolves elsewhere than expected.

Common situations: Reading output of a job that failed or wrote to a different directory; wrong bucket/prefix in the URI; eventual-consistency lag between write and read in other stores (TOS is strong-consistent, so this usually means a genuine typo/path bug); relative paths resolved against /user/<name> working directory.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• File not found %s
• {path}
• {path}: No such file or directory!
• key + ": No such file or directory."
• No such file or directory '{}'

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d7c2ec051e992dfb. Report an issue: GitHub.