What causes "No such file or directory: {}" in apache/hadoop?

Pre-check existence with fs.exists(path)/fs.getFileStatus(path) before operations that require the directory, and treat 'missing' as already-done for idempotent deletes.

apache/hadoop · error · FileNotFoundException

No such file or directory: {}

Error message

No such file or directory: {}

What it means

In OBSCommonUtils' directory-emptiness check, after listing finds no children and the key is neither root nor an fs(POSIX)-bucket folder marker, the method concludes the directory does not exist and throws FileNotFoundException('No such file or directory: <obsKey>') where obsKey is the raw object key (not a user-facing path). For object-layout buckets, an 'empty directory' cannot be distinguished from a missing one, so absence is surfaced as FNFE during operations (e.g. delete of an empty dir) that expected the directory to exist.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSCommonUtils.java:980

          LOG.debug("Summary: {} {}", summary.getObjectKey(),
              summary.getMetadata().getContentLength());
        }
        for (String prefix : objects.getCommonPrefixes()) {
          LOG.debug("Prefix: {}", prefix);
        }
      }
      LOG.debug("Found non-empty directory {}", obsKey);
      return false;
    } else if (obsKey.isEmpty()) {
      LOG.debug("Found root directory");
      return true;
    } else if (owner.isFsBucket()) {
      LOG.debug("Found empty directory {}", obsKey);
      return true;
    }

    LOG.debug("Not Found: {}", obsKey);
    throw new FileNotFoundException("No such file or directory: " + obsKey);
  }

  /**
   * Build a {@link LocatedFileStatus} from a {@link FileStatus} instance.
   *
   * @param owner  the owner OBSFileSystem instance
   * @param status file status
   * @return a located status with block locations set up from this FS.
   * @throws IOException IO Problems.
   */
  static LocatedFileStatus toLocatedFileStatus(final OBSFileSystem owner,
      final FileStatus status) throws IOException {
    return new LocatedFileStatus(
        status, status.isFile() ? owner.getFileBlockLocations(status, 0,
        status.getLen()) : null);
  }

  /**

View on GitHub (pinned to 2add963021)

Solutions

Pre-check existence with fs.exists(path)/fs.getFileStatus(path) before operations that require the directory, and treat 'missing' as already-done for idempotent deletes.
When creating directories you will later manage, ensure the connector actually materializes markers: use fs (POSIX) buckets or write explicit 0-byte markers for object buckets.
Catch FileNotFoundException (and FileNotFound subclasses) around cleanup and convert to a no-op success when absence equals the desired end state.
Audit for trailing-slash or encoding mismatches between the creating and deleting code paths.

Example fix

// before
fs.delete(new Path("obs://b/staging/"), false);
// -> FileNotFoundException: No such file or directory: staging/

// after
Path p = new Path("obs://b/staging");
try {
  fs.delete(p, false);
} catch (FileNotFoundException e) {
  LOG.info("{} already absent; treating delete as success", p);
}

Defensive patterns

Strategy: try-catch

Validate before calling

static void deleteDirIfPresent(FileSystem fs, Path p) throws IOException {
  if (!fs.exists(p)) { LOG.debug("{} already absent", p); return; }
  fs.delete(p, false);
}

Try / catch

try {
  fs.delete(dirPath, false);
} catch (FileNotFoundException e) {
  LOG.info("{} vanished before delete (race or never created) — treating as success", dirPath);
}

Prevention

Make deletes idempotent: exists() check or catch FNFE and treat as success.
Prefer fs (POSIX) buckets when your logic depends on real empty-directory markers.
Avoid trailing-slash variants of the same path across producers and consumers; pick one canonical form.

When it happens

Trigger: delete('obs://b/emptydir', false) or similar when 'emptydir' was never created (no 0-byte dir marker in an object bucket); racing writers where a sibling removes the marker between list and check; operations on POSIX buckets that failed to create the folder marker; keys with trailing-slash confusion where the marker was written under a slightly different name.

Common situations: Idempotent cleanup code deleting dirs that may not exist; POSIX vs object bucket behavior differences (fs buckets keep real folder markers, object buckets infer dirs from children); eventual-consistency windows after concurrent deletes; encoded characters producing different marker keys.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0f5d401d66936bf5. Report an issue: GitHub.