apache/hadoop · error · FileNotFoundException

Path {} does not exist

Error message

Path {} does not exist

What it means

While assembling a batched listing, getListingInt returned null for one of the source paths, which in HDFS semantics means the path does not exist. FileNotFoundException names the offending src and aborts the whole batch request.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:4358

    checkOperation(OperationCategory.READ);
    readLock(RwLockMode.FS);
    try {
      checkOperation(NameNode.OperationCategory.READ);

      // List all directories from the starting index until we've reached
      // ls limit OR finished listing all srcs.
      LinkedHashMap<Integer, HdfsPartialListing> listings =
          Maps.newLinkedHashMap();
      DirectoryListing lastListing = null;
      int numEntries = 0;
      for (; srcsIndex < srcs.length; srcsIndex++) {
        String src = srcs[srcsIndex];
        HdfsPartialListing listing;
        try {
          DirectoryListing dirListing =
              getListingInt(dir, pc, src, indexStartAfter, needLocation);
          if (dirListing == null) {
            throw new FileNotFoundException("Path " + src + " does not exist");
          }
          if (needLocation && isObserver()) {
            for (HdfsFileStatus fs : dirListing.getPartialListing()) {
              if (fs instanceof HdfsLocatedFileStatus) {
                LocatedBlocks lbs = ((HdfsLocatedFileStatus) fs).getLocatedBlocks();
                checkBlockLocationsWhenObserver(lbs, fs.toString());
              }
            }
          }
          listing = new HdfsPartialListing(
              srcsIndex, Lists.newArrayList(dirListing.getPartialListing()));
          numEntries += listing.getPartialListing().size();
          lastListing = dirListing;
        } catch (Exception e) {
          if (e instanceof ObserverRetryOnActiveException) {
            throw (ObserverRetryOnActiveException) e;
          }
          if (e instanceof AccessControlException) {

View on GitHub (pinned to 2add963021)

Solutions

1. Pre-validate each src with fs.exists (or util.exists) before submitting the batch
2. Handle FileNotFoundException per-batch by dropping the missing path and re-issuing the rest
3. Snapshot the path list once and tolerate stale entries rather than mixing live lookups

Example fix

// before
String[] srcs = requestedPaths.toArray(new String[0]);

// after
List<String> srcs = new ArrayList<>();
for (String p : requestedPaths) {
  if (fs.util().exists(new Path(p))) { srcs.add(p); }
  else { LOG.debug("Skipping vanished path: " + p); }
}

List<String> valid = new ArrayList<>();
for (String s : srcs) {
  if (fs.util().exists(new Path(s))) { valid.add(s); }
}

try {
  listing = getBatchedListing(srcs, startAfter, needLocation);
} catch (FileNotFoundException e) {
  // drop the vanished path named in the message and re-issue the batch
}

When it happens

Trigger: A path in srcs was deleted or renamed between pagination calls (startAfter cursor) or never existed (typo, wrong mount); listing a path removed after the batch was assembled.

Common situations: Concurrent deletions during a long cursor-based listing; directory trees being restructured while enumerated; jobs listing dynamic directories that appear and disappear.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• "File/Directory " + iip.getPath() + " does not exist."
• File is deleted: {} (inode {}) {}
• File not found: {}, likely due to delayed block removal
• Too many source paths (%d > %d)
• Directory does not exist: {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/49b0eaf116b010fd. Report an issue: GitHub.