apache/hadoop · error · IOException

Failed to delete {bpCurrentDir}

Error message

Failed to delete {bpCurrentDir}

What it means

Thrown by FsVolumeImpl.deleteBPDirectories() (FsVolumeImpl.java:1198) when FileIoProvider.delete() fails on the block pool's current/ directory itself after every listed child was supposedly removed. It means the directory was not empty at unlink time (a file or subdir appeared between the loop and this call, or listFiles() hid an undeletable entry) or the filesystem refused the rmdir (permissions, read-only, NFS).

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java:1198

              this, finalizedDir, fileIoProvider) ||
          !fileIoProvider.fullyDelete(
              this, finalizedDir)) {
        throw new IOException("Failed to delete " + finalizedDir);
      }
      if (lazypersistDir.exists() &&
          ((!DatanodeUtil.dirNoFilesRecursive(
              this, lazypersistDir, fileIoProvider) ||
              !fileIoProvider.fullyDelete(this, lazypersistDir)))) {
        throw new IOException("Failed to delete " + lazypersistDir);
      }
      fileIoProvider.fullyDelete(this, tmpDir);
      for (File f : fileIoProvider.listFiles(this, bpCurrentDir)) {
        if (!fileIoProvider.delete(this, f)) {
          throw new IOException("Failed to delete " + f);
        }
      }
      if (!fileIoProvider.delete(this, bpCurrentDir)) {
        throw new IOException("Failed to delete " + bpCurrentDir);
      }
      for (File f : fileIoProvider.listFiles(this, bpDir)) {
        if (!fileIoProvider.delete(this, f)) {
          throw new IOException("Failed to delete " + f);
        }
      }
      if (!fileIoProvider.delete(this, bpDir)) {
        throw new IOException("Failed to delete " + bpDir);
      }
    }
  }

  @Override
  public String getStorageID() {
    return storageID;
  }

  @Override

View on GitHub (pinned to 2add963021)

Solutions

1. Confirm nothing recreated files under bpDir/current (ls -la), stop the writer, then retry deleteBlockPool
2. Fix ownership of the directory itself, not just children: chown <dn-user> <bpDir>/current, then retry
3. Check the mount is read-write and healthy (mount, dmesg | grep -i error)
4. Use force mode if the residue is known: hdfs dfsadmin -deleteBlockPool <dn> <bpid> force
5. As a last resort stop the DataNode, rm -rf the block pool dir manually, restart

Example fix

# before: current/ dir itself undeletable, IOException aborts cleanup
hdfs dfsadmin -deleteBlockPool dn1.hadoop:9866 BP-1234567890

# after: verify empty, fix dir ownership, force delete
ls -la /data/dfs/dn/current/BP-1234567890/current
chown hdfs:hdfs /data/dfs/dn/current/BP-1234567890/current
hdfs dfsadmin -deleteBlockPool dn1.hadoop:9866 BP-1234567890 force

File current = new File(bpDir, "current");
// re-check emptiness immediately before the delete call, not minutes before
boolean empty = current.isDirectory()
    && current.list() != null && current.list().length == 0;
if (!empty) {
  LOG.warn("current/ not empty at delete time for bp {}", bpid);
}

try {
  volume.deleteBPDirectories(bpid, false);
} catch (IOException e) {
  if (e.getMessage().contains("current")) {
    // list what reappeared, stop the writer, then retry; else force-delete
    LOG.warn("current/ delete failed: {}", e);
  }
}

When it happens

Trigger: deleteBlockPool (non-force) racing with a thread recreating a file inside current/; a hidden/subdirectory entry the child loop could not remove (its own 'Failed to delete {f}' would normally fire first); NFS or FUSE semantics where rmdir of a busy dir returns EACCES/EBUSY.

Common situations: Delete issued while the DataNode is still finalizing writes for that block pool; data dirs on NFS with silly-rename; wrong ownership of the current/ directory inode itself; disk remounted read-only after errors.

Related errors

• Failed to delete {lazypersistDir}
• Failed to delete {f}
• Failed to delete {bpDir}
• Cannot remove directory {}
• Failed to rename {} to {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/52bad725fa98111d. Report an issue: GitHub.