apache/hadoop · error · DirectoryNotEmptyException

Cannot delete a non-empty directory. : %s

Error message

Cannot delete a non-empty directory. : %s

What it means

delete(path, recursive=false) lists the directory (listDirectory) and throws DirectoryNotEmptyException if any children exist — objects under the prefix and/or explicit directory-marker objects. Recursive delete uses listRecursive and removes everything instead. This mirrors the Hadoop FileSystem.delete(non-recursive) contract: fail rather than silently partial-delete.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageFileSystem.java:254

  void delete(URI path, boolean recursive) throws IOException {
    checkNotNull(path, "path should not be null");
    checkArgument(!path.equals(GCSROOT), "Cannot delete root path (%s)", path);

    FileInfo fileInfo = getFileInfo(path);
    if (!fileInfo.exists()) {
      throw new FileNotFoundException("Item not found: " + path);
    }

    List<FileInfo> itemsToDelete;
    // Delete sub-items if it is a directory.
    if (fileInfo.isDirectory()) {
      itemsToDelete =
          recursive
              ? listRecursive(fileInfo.getPath()) // TODO: Get only one result
              : listDirectory(fileInfo.getPath());

      if (!itemsToDelete.isEmpty() && !recursive) {
        throw new DirectoryNotEmptyException("Cannot delete a non-empty directory. : " + path);
      }
    } else {
      itemsToDelete = new ArrayList<>();
    }

    List<FileInfo> bucketsToDelete = new ArrayList<>();
    (fileInfo.getItemInfo().isBucket() ? bucketsToDelete : itemsToDelete).add(fileInfo);

    deleteObjects(itemsToDelete, bucketsToDelete);

    StorageResourceId parentId =
        StorageResourceId.fromUriPath(UriPaths.getParentPath(path), true);
    GoogleCloudStorageItemInfo parentInfo =
        getFileInfoInternal(parentId, /* inferImplicitDirectories= */ false);

    StorageResourceId resourceId = parentInfo.getResourceId();
    if (parentInfo.exists()
        || resourceId.isRoot()

View on GitHub (pinned to 2add963021)

Solutions

1. Pass recursive=true when the intent is to remove the directory and its contents.
2. If the non-recursive contract matters, first list the directory and surface a precise 'not empty: <children>' error.
3. Check for hidden children (_SUCCESS, .crc, empty markers) before concluding a directory should be deletable.

Example fix

// before
fs.delete(dirPath, false); // DirectoryNotEmptyException

// after
FileStatus[] children = fs.listStatus(dirPath);
if (children.length == 0) {
  fs.delete(dirPath, false);
} else {
  fs.delete(dirPath, true); // or fail with a domain-specific message
}

if (fileInfo.isDirectory() && !recursive) {
  List<FileInfo> children = gcsFs.listDirectory(fileInfo.getPath());
  if (!children.isEmpty()) {
    throw new NonEmptyDirectoryException(path, children); // actionable message
  }
}
gcsFs.delete(path, recursive);

catch (DirectoryNotEmptyException e) {
  // decide policy: escalate to recursive delete, or report children to the user
}

When it happens

Trigger: FileSystem.delete(path, false) on a GCS directory containing anything: real data objects, _SUCCESS/.crc markers, or empty directory placeholders from mkdirs.

Common situations: Job cleanup calling delete(dir, false) as an existence/emptiness check; accidental non-recursive flag; a directory that 'looks empty' but hides _SUCCESS, .crc, or dot-files.

Related errors

• Can not delete the directory: [%s], as it is not empty and o
• Deleting resource %s failed.
• Item not found: %s
• {path}
• {path}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3dcb1bcf970ccbbf. Report an issue: GitHub.