What causes "Directory {} is not empty." in apache/hadoop?

Call delete(path, true) to delete the directory recursively

apache/hadoop · error · IOException

Directory {} is not empty.

Error message

Directory {} is not empty.

What it means

Thrown by HierarchyBosNativeFileSystemStore.deleteDirs when a non-recursive delete is requested on a directory that still contains entries. The store lists up to 1 child under the key and, if any file or subdirectory is found, refuses the delete with IOException 'Directory <key> is not empty.' This mirrors the POSIX rmdir/HDFS semantics on the BOS hierarchy store.

Source

Thrown at hadoop-cloud-storage-project/hadoop-bos/src/main/java/org/apache/hadoop/fs/bos/HierarchyBosNativeFileSystemStore.java:115

  protected boolean isHierarchy() {
    return true;
  }

  /**
   * {@inheritDoc}
   *
   * Deletes a directory. For non-recursive deletes, verifies
   * the directory is empty first.
   */
  @Override
  public void deleteDirs(
      String key, boolean recursive) throws IOException {
    if (!recursive) {
      PartialListing listing =
          list(key, 1, null, "/");
      if (listing.getFiles().length
          + listing.getDirectories().length > 0) {
        throw new IOException(
            "Directory " + key + " is not empty.");
      }
      delete(key);
    } else {
      String marker = null;
      DeleteDirectoryResponse response = null;
      do {
        response = bosClientProxy.deleteDirectory(
            bucketName, key, true, marker);
        if (response != null
            && response.isTruncated()) {
          marker = response.getNextDeleteMarker();
        } else {
          break;
        }
      } while (response.isTruncated());
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

Call delete(path, true) to delete the directory recursively
List children first (fs.listStatus(path)) and delete them explicitly if you truly need per-entry control
Guard the non-recursive call with an emptiness check and surface a clear message to the caller

Example fix

// before
fs.delete(new Path("bosn://bucket/out"), false); // throws: dir not empty
// after
Path dir = new Path("bosn://bucket/out");
if (fs.getFileStatus(dir).isDirectory() && fs.listStatus(dir).length > 0) {
  fs.delete(dir, true);
} else {
  fs.delete(dir, false);
}

Defensive patterns

Strategy: validation

Validate before calling

Path dir = new Path("bosn://bucket/out");
if (!fs.getFileStatus(dir).isDirectory()
    || fs.listStatus(dir).length == 0) {
  fs.delete(dir, false); // safe: absent or empty
} else {
  fs.delete(dir, true);
}

Try / catch

catch (IOException e) { if (e.getMessage().contains("is not empty")) { /* retry recursive or report */ } else throw e; }

Prevention

Default to recursive delete for object-store directories
Treat non-recursive delete as an assertion of emptiness, not a cleanup call

When it happens

Trigger: Calling FileSystem.delete(path, false) on a BOS directory whose listing returns at least one file or subdirectory; e.g. cleanup code that deletes job output directories non-recursively, or exists()+delete() sequences copied from local-FileSystem code.

Common situations: Job cleanup after a failed run leaves _temporary subdirectories; distcp or Hive scripts that issue non-recursive deletes of directories that were populated concurrently; code ported from a filesystem where delete() silently recursed.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9471f95e8f5d5437. Report an issue: GitHub.