apache/hadoop · error · FileNotFoundException

File does not exist: ${f}

Error message

File does not exist: ${f}

What it means

getHdfsFileStatus issues WebHDFS GETFILESTATUS and decodes the response into an HdfsFileStatus. If decoding yields null, most often because the response body is empty or contains no status object, the client throws FileNotFoundException with the requested path. This also covers the ordinary case where the path is not present in the namespace.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java:1116

  private FsPermission applyUMask(FsPermission permission) {
    if (permission == null) {
      permission = FsPermission.getDefault();
    }
    return FsCreateModes.applyUMask(permission,
        FsPermission.getUMask(getConf()));
  }

  private HdfsFileStatus getHdfsFileStatus(Path f) throws IOException {
    final HttpOpParam.Op op = GetOpParam.Op.GETFILESTATUS;
    HdfsFileStatus status = new FsPathResponseRunner<HdfsFileStatus>(op, f) {
      @Override
      HdfsFileStatus decodeResponse(Map<?,?> json) {
        return JsonUtilClient.toFileStatus(json, true);
      }
    }.run();
    if (status == null) {
      throw new FileNotFoundException("File does not exist: " + f);
    }
    return status;
  }

  @Override
  public FileStatus getFileStatus(Path f) throws IOException {
    statistics.incrementReadOps(1);
    storageStatistics.incrementOpCounter(OpType.GET_FILE_STATUS);
    return getHdfsFileStatus(f).makeQualified(getUri(), f);
  }

  @Override
  public AclStatus getAclStatus(Path f) throws IOException {
    final HttpOpParam.Op op = GetOpParam.Op.GETACLSTATUS;
    AclStatus status = new FsPathResponseRunner<AclStatus>(op, f) {
      @Override
      AclStatus decodeResponse(Map<?,?> json) {
        return JsonUtilClient.toAclStatus(json);

View on GitHub (pinned to 2add963021)

Solutions

1. Log the qualified path, fs.makeQualified(path), and compare it with the path shown in the exception.
2. Create or wait for the parent/file to be committed before requesting status.
3. Use explicit absolute paths in jobs to avoid dependence on the client working directory.
4. If deletion or rename can race with the caller, treat FileNotFoundException as an expected outcome rather than retrying the same operation.

Example fix

// before
FileStatus status = fs.getFileStatus(new Path("input/data.csv"));

// after
Path f = fs.makeQualified(new Path("/user/alice/input/data.csv"));
try {
  return fs.getFileStatus(f);
} catch (FileNotFoundException e) {
  LOG.info("Input not committed yet: {}", f);
  return null;
}

Path qualified = fs.makeQualified(path);
if (!fs.exists(qualified)) {
  throw new FileNotFoundException("Expected input is missing: " + qualified);
}

try {
  return fs.getFileStatus(f);
} catch (FileNotFoundException e) {
  if (optionalInputs.contains(f)) {
    return null;
  }
  throw new MissingInputException("Required HDFS input does not exist: " + f, e);
}

When it happens

Trigger: Calling getFileStatus(path) on a webhdfs:// path that does not exist, or when the NameNode/gateway returns an empty body for GETFILESTATUS. Relative paths are resolved against the current working directory, so the effective path may differ from the string supplied by the application.

Common situations: Reading before a writer creates or commits the file; wrong working directory; typo or missing leading slash; a path created under another user's namespace; races with delete/rename.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Invalid DFS directory name ${result}
• The length to read ${length} exceeds the file length ${fin.l
• End of file reached before reading fully.
• Invalid value in server response: name=[${name}]
• Missing both 'ipAddr' and 'name' in server response.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/404e7ff966048e96. Report an issue: GitHub.