apache/hadoop · error · FileNotFoundException

No such file or directory: {}

Error message

No such file or directory: {}

What it means

In the object-bucket getFileStatus, a path is probed as an object and as a directory; when the underlying ObsException carries response code 404 and the folder is empty-check also misses, the code throws FileNotFoundException('No such file or directory: <path>'). This matches the Hadoop contract that getFileStatus fails with FNFE for absent paths.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSObjectBucketUtils.java:813

                newKey, e);
          }
        }
      }
    }

    try {
      boolean isEmpty = OBSCommonUtils.innerIsFolderEmpty(owner, key);
      LOG.debug("Is dir ({}) empty? {}", path, isEmpty);
      return new OBSFileStatus(path, owner.getUsername());
    } catch (ObsException e) {
      if (e.getResponseCode() != OBSCommonUtils.NOT_FOUND_CODE) {
        throw OBSCommonUtils.translateException("getFileStatus", key,
            e);
      }
    }

    LOG.debug("Not Found: {}", path);
    throw new FileNotFoundException("No such file or directory: " + path);
  }

  static ContentSummary getDirectoryContentSummary(final OBSFileSystem owner,
      final String key) throws IOException {
    String newKey = key;
    newKey = OBSCommonUtils.maybeAddTrailingSlash(newKey);
    long[] summary = {0, 0, 1};
    LOG.debug("Summary key {}", newKey);
    ListObjectsRequest request = new ListObjectsRequest();
    request.setBucketName(owner.getBucket());
    request.setPrefix(newKey);
    Set<String> directories = new TreeSet<>();
    request.setMaxKeys(owner.getMaxKeys());
    ObjectListing objects = OBSCommonUtils.listObjects(owner, request);
    while (true) {
      if (!objects.getCommonPrefixes().isEmpty() || !objects.getObjects()
          .isEmpty()) {
        if (LOG.isDebugEnabled()) {

View on GitHub (pinned to 2add963021)

Solutions

  1. Check fs.exists(path) or catch FileNotFoundException when absence is an expected outcome
  2. Verify the path string, bucket, and prefix against an actual listing (fs.listStatus on the parent)
  3. For writer/reader races, wait on a completion marker such as _SUCCESS instead of retrying blindly

Example fix

// before
FSDataInputStream in = fs.open(new Path("/data/part-0"));

// after
Path p = new Path("/data/part-0");
if (!fs.exists(p)) {
  throw new FileNotFoundException("Input missing: " + p);
}
FSDataInputStream in = fs.open(p);
Defensive patterns

Strategy: try-catch

Validate before calling

if (!fs.exists(p)) {
  // handle absence before calling open/getFileStatus
  return Optional.empty();
}

Try / catch

try {
  return Optional.of(fs.getFileStatus(p));
} catch (FileNotFoundException e) {
  return Optional.empty(); // absence is a normal outcome for lookups
}

Prevention

When it happens

Trigger: fs.open or fs.getFileStatus on a path never written, deleted by another client, or whose parent directory has no marker object; also paths with stray trailing slashes or double slashes that map to a different object key.

Common situations: Race between a writer and a reader on OBS; wrong bucket or prefix in the job configuration; cleanup removed data before a late task read it; a straggler task reading a source already renamed away.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4ac3b1ea6bb65b3d. Report an issue: GitHub.