apache/hadoop · error · IOException

Can not delete the directory: [%s], as it is not empty and o

Error message

Can not delete the directory: [%s], as it is not empty and option recursive is false.

What it means

delete(f, false) on a directory that contains at least one entry throws IOException stating the directory is not empty and recursive is false. CosN lists the directory via listStatus before deleting, and any file or common prefix blocks a non-recursive delete, matching POSIX rmdir semantics rather than the silent prefix-delete some object stores offer.

Source

Thrown at hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNFileSystem.java:318

      LOG.debug("Ready to delete the file: [{}], but it does not exist.", f);
      return false;
    }
    Path absolutePath = makeAbsolute(f);
    String key = pathToKey(absolutePath);
    if (key.compareToIgnoreCase("/") == 0) {
      FileStatus[] fileStatuses = listStatus(f);
      return this.rejectRootDirectoryDelete(
          fileStatuses.length == 0, recursive);
    }

    if (status.isDirectory()) {
      if (!key.endsWith(PATH_DELIMITER)) {
        key += PATH_DELIMITER;
      }
      if (!recursive && listStatus(f).length > 0) {
        String errMsg = String.format("Can not delete the directory: [%s], as"
            + " it is not empty and option recursive is false.", f);
        throw new IOException(errMsg);
      }

      createParent(f);

      String priorLastKey = null;
      do {
        PartialListing listing = store.list(
            key,
            Constants.COS_MAX_LISTING_LENGTH,
            priorLastKey,
            true);
        for (FileMetadata file : listing.getFiles()) {
          store.delete(file.getKey());
        }
        for (FileMetadata commonPrefix : listing.getCommonPrefixes()) {
          store.delete(commonPrefix.getKey());
        }
        priorLastKey = listing.getPriorLastKey();

View on GitHub (pinned to 2add963021)

Solutions

1. Delete recursively: fs.delete(dir, true) or hadoop fs -rm -r.
2. If selective cleanup is needed, list children and delete them explicitly, then remove the directory.
3. If the directory should be empty, run fs.listStatus(dir) to find leftover entries (_SUCCESS, .crc, markers) and remove them before retrying.

Example fix

// before
fs.delete(dir, false); // IOException: ... not empty and option recursive is false

// after
boolean ok = fs.delete(dir, true);
// or explicit:
for (FileStatus st : fs.listStatus(dir)) { fs.delete(st.getPath(), true); }
ok = fs.delete(dir, false);

if (fs.getFileStatus(dir).isDirectory() && !recursive) {
  FileStatus[] children = fs.listStatus(dir);
  if (children.length > 0) {
    // decide: recursive delete, explicit cleanup, or skip
    for (FileStatus st : children) { fs.delete(st.getPath(), true); }
  }
}

try {
  fs.delete(dir, false);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains('not empty')) {
    fs.delete(dir, true);
  } else { throw e; }
}

When it happens

Trigger: fs.delete(dir, false) when listStatus(dir).length > 0; hadoop fs -rm cosn://bucket/dir (no -r) on a populated directory; 'hidden' children such as _SUCCESS, .crc files, or 0-byte directory markers making an apparently empty directory non-empty.

Common situations: Attempting rmdir where a previous job left _SUCCESS or part files; directory markers left by other COS tools; porting code from an object store whose delete on a prefix silently succeeded.

Related errors

• Can not delete root path
• {path}
• {path}
• Directory " + f.toString() + " is not empty
• Directory is not empty

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ef32056648d681e4. Report an issue: GitHub.