What causes "File does not exist: {}" in apache/hadoop?

Verify the exact path on the same cluster and user: hdfs dfs -ls ' '

apache/hadoop · error · FileNotFoundException

File does not exist: {}

Error message

File does not exist: {}

What it means

GET op=GETFILESTATUS handler. ClientProtocol.getFileInfo returns null — not an exception — when the path is absent from the namespace, so the WebHDFS layer converts null into FileNotFoundException('File does not exist: <fullpath>'), which reaches the REST client as HTTP 404 with a RemoteException body. The path in the message is the decoded full path that reached the NameNode.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java:1409

      BlockLocation[] locations =
          DFSUtilClient.locatedBlocks2Locations(locatedBlocks);
      final String js = JsonUtil.toJsonString(locations);
      return Response.ok(js).type(MediaType.APPLICATION_JSON).build();
    }
    case GET_BLOCK_LOCATIONS:
    {
      final long offsetValue = offset.getValue();
      final Long lengthValue = length.getValue();
      final LocatedBlocks locatedblocks = cp.getBlockLocations(fullpath,
          offsetValue, lengthValue != null? lengthValue: Long.MAX_VALUE);
      final String js = JsonUtil.toJsonString(locatedblocks);
      return Response.ok(js).type(MediaType.APPLICATION_JSON).build();
    }
    case GETFILESTATUS:
    {
      final HdfsFileStatus status = cp.getFileInfo(fullpath);
      if (status == null) {
        throw new FileNotFoundException("File does not exist: " + fullpath);
      }

      final String js = JsonUtil.toJsonString(status, true);
      return Response.ok(js).type(MediaType.APPLICATION_JSON).build();
    }
    case LISTSTATUS:
    {
      final StreamingOutput streaming = getListingStream(cp, fullpath);
      return Response.ok(streaming).type(MediaType.APPLICATION_JSON).build();
    }
    case GETCONTENTSUMMARY:
    {
      final ContentSummary contentsummary = cp.getContentSummary(fullpath);
      final String js = JsonUtil.toJsonString(contentsummary);
      return Response.ok(js).type(MediaType.APPLICATION_JSON).build();
    }
    case GETQUOTAUSAGE:
    {

View on GitHub (pinned to 2add963021)

Solutions

Verify the exact path on the same cluster and user: hdfs dfs -ls '<path>'
Percent-encode every path segment and strip whitespace/case-fix before sending
If deletion races are expected, treat the 404 as 'gone' and re-check the parent listing instead of retrying blindly
For a symlink's own status use op=GETFILELINKSTATUS; GETFILESTATUS follows the link

Example fix

// before: assumes the file still exists
HdfsFileStatus s = webhdfs.getClient().getFileInfo(path);

// after: 404 is a normal outcome when paths race with cleanup
try {
  FileStatus s = webhdfs.getFileStatus(new Path(path));
  return Optional.of(s);
} catch (FileNotFoundException e) {
  return Optional.empty(); // deleted between listing and fetch
}

Defensive patterns

Strategy: try-catch

Validate before calling

null

Try / catch

try {
  FileStatus s = webhdfs.getFileStatus(new Path(path));
  return Optional.of(s);
} catch (FileNotFoundException e) { // server returned 404
  return Optional.empty(); // absent is a valid answer — do not retry the same path
}

Prevention

Percent-encode every path segment; never concatenate raw user input into the URL
In polling loops, treat 404 as a terminal state for that path rather than an error
Prefer catching FileNotFoundException over exists() pre-checks — the pre-check costs the same RPC and still races

When it happens

Trigger: op=GETFILESTATUS on a deleted or never-created path; a path with wrong casing, a trailing space, or broken percent-encoding (unencoded %, #, ? or space); a race where another client renames/deletes the file between your listing and this status call.

Common situations: Monitor/polling loops racing with job cleanup that deletes outputs; doAs/user mismatch resolving a different home directory; copy-pasted URLs from logs with partial encoding; case-sensitive path mistakes after migrating from a case-insensitive filesystem.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8418f2b8f44b9c47. Report an issue: GitHub.