apache/hadoop · error · IOException

Directory " + f.toString() + " is not empty

Error message

Directory " + f.toString() + " is not empty

What it means

RawLocalFileSystem.delete(Path, recursive=false) throws IOException('Directory ... is not empty') when the target is a directory containing entries. Non-recursive delete is deliberately safe: it removes only empty directories and files. The check uses FileUtil.listFiles, so even hidden dotfiles count as content.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:800

  /**
   * Delete the given path to a file or directory.
   * @param p the path to delete
   * @param recursive to delete sub-directories
   * @return true if the file or directory and all its contents were deleted
   * @throws IOException if p is non-empty and recursive is false 
   */
  @Override
  public boolean delete(Path p, boolean recursive) throws IOException {
    File f = pathToFile(p);
    if (!f.exists()) {
      //no path, return false "nothing to delete"
      return false;
    }
    if (f.isFile()) {
      return f.delete();
    } else if (!recursive && f.isDirectory() && 
        (FileUtil.listFiles(f).length != 0)) {
      throw new IOException("Directory " + f.toString() + " is not empty");
    }
    return FileUtil.fullyDelete(f);
  }
 
  /**
   * {@inheritDoc}
   *
   * (<b>Note</b>: Returned list is not sorted in any given order,
   * due to reliance on Java's {@link File#list()} API.)
   */
  @Override
  public FileStatus[] listStatus(Path f) throws IOException {
    File localf = pathToFile(f);
    FileStatus[] results;

    if (!localf.exists()) {
      throw new FileNotFoundException("File " + f + " does not exist");
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Pass recursive=true when full cleanup is intended: fs.delete(dir, true).
2. List first if selective deletion is needed: delete children individually, then remove the (now empty) directory with recursive=false.
3. Check for hidden files: ls -la the directory; remove .crc/_SUCCESS/_temporary leftovers explicitly.
4. Handle the race by catching IOException and re-checking emptiness rather than blindly retrying.

Example fix

// before
fs.delete(new Path("/out"), false); // throws: contains part files / .crc files

// after
boolean removed = fs.delete(new Path("/out"), true);

if (fs.getFileStatus(dir).isDirectory()
      && fs.listStatus(dir).length > 0 && !recursive) {
  throw new IllegalStateException(dir + " not empty; pass recursive=true or clear it");
}
fs.delete(dir, recursive);

try {
  fs.delete(dir, false);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("not empty")) {
    fs.delete(dir, true); // fall back to recursive when full cleanup is safe
  } else {
    throw e;
  }
}

When it happens

Trigger: Calling fs.delete(dirPath, false) on a directory with any children: cleaning a job output directory non-recursively, attempting to remove a directory that contains hidden .tmp/.crc files, or racing with a writer that added a file between listing and delete.

Common situations: Cleanup code that assumed the directory was empty, Hadoop's per-file .crc checksum sidecar files (e.g. .part-00000.crc) keeping a 'visually empty' directory non-empty, or temp directories with dot-prefixed files.

Related errors

• Can not delete the directory: [%s], as it is not empty and o
• {path}
• {path}
• Har: delete not allowed
• Failed to delete {pTask}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/63b08772062279c9. Report an issue: GitHub.