apache/hadoop · error · IOException

Cannot remove directory {}

Error message

Cannot remove directory {}

What it means

During the same legacy detach cleanup, the detached/ directory was empty but File.delete() returned false — the directory could not be removed, so the upgrade still fails. Usually a permissions/ownership problem, an open handle, or exotic filesystem (NFS) semantics.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java:541

   * 
   * If the directory is not empty report an error; Otherwise remove the
   * directory.
   * 
   * @param detachDir detach directory
   * @throws IOException if the directory is not empty or it can not be removed
   */
  private void cleanupDetachDir(File detachDir) throws IOException {
    if (!DataNodeLayoutVersion.supports(
        LayoutVersion.Feature.APPEND_RBW_DIR, layoutVersion)
        && detachDir.exists() && detachDir.isDirectory()) {

      if (FileUtil.list(detachDir).length != 0) {
        throw new IOException("Detached directory " + detachDir
            + " is not empty. Please manually move each file under this "
            + "directory to the finalized directory if the finalized "
            + "directory tree does not have the file.");
      } else if (!detachDir.delete()) {
        throw new IOException("Cannot remove directory " + detachDir);
      }
    }
  }

  /**
   * Restore all files from the trash directory to their corresponding
   * locations under current/
   */
  private int restoreBlockFilesFromTrash(File trashRoot)
      throws  IOException {
    int filesRestored = 0;
    File[] children = trashRoot.exists() ? trashRoot.listFiles() : null;
    if (children == null) {
      return 0;
    }

    File restoreDirectory = null;
    for (File child : children) {

View on GitHub (pinned to 2add963021)

Solutions

1. Check ownership/permissions of detached/ and its parent (chown to the DN user)
2. Find and stop processes holding the directory (lsof), ensure the mount is read-write
3. Remove detached/ manually, then restart the upgrade

Example fix

chown -R hdfs:hdfs /dfs/dn/current/BP-*/detached
rmdir /dfs/dn/current/BP-*/detached
hdfs --daemon start datanode -upgrade

if (!detachDir.isDirectory() || Files.isWritable(detachDir.getParentFile().toPath())) {
  // ok to attempt cleanup
}

catch (IOException e) {
  if (e.getMessage().contains("Cannot remove directory")) {
    // permissions/handle issue: surface actionable context, do not loop retries
    throw new IOException("Cannot remove " + dir + ": check ownership/open handles", e);
  }
}

When it happens

Trigger: DN process lacks write permission on the parent of detached/; another process (or NFS client) holds the directory; read-only mount.

Common situations: Running the DN as a different user than the one owning the storage dirs; NAS-backed data dirs; containers with restricted mounts.

Related errors

• Cannot remove directory {}
• Failed to rename {} to {}
• Storage directory with location {} does not exist
• Cannot create directory {}
• Mkdirs failed to create {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/473be5cfcd5c75eb. Report an issue: GitHub.