apache/hadoop · error · IOException

Destination '{destFile}' exists but cannot be deleted

Error message

Destination '{destFile}' exists but cannot be deleted

What it means

Guard clause of Storage.nativeCopyFileUnbuffered: destFile exists, passes the canWrite check, but File.delete() still returned false. Deletion can fail even on writable files when the file is held open by another process (notably on Windows), the containing directory's permissions prevent unlink, or SELinux/ACLs deny removal.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java:1355

    }
    if (srcFile.getCanonicalPath().equals(destFile.getCanonicalPath())) {
      throw new IOException("Source '" + srcFile + "' and destination '" +
          destFile + "' are the same");
    }
    File parentFile = destFile.getParentFile();
    if (parentFile != null) {
      if (!parentFile.mkdirs() && !parentFile.isDirectory()) {
        throw new IOException("Destination '" + parentFile
            + "' directory cannot be created");
      }
    }
    if (destFile.exists()) {
      if (FileUtil.canWrite(destFile) == false) {
        throw new IOException("Destination '" + destFile
            + "' exists but is read-only");
      } else {
        if (destFile.delete() == false) {
          throw new IOException("Destination '" + destFile
              + "' exists but cannot be deleted");
        }
      }
    }
    try {
      NativeIO.copyFileUnbuffered(srcFile, destFile);
    } catch (NativeIOException e) {
      throw new IOException("Failed to copy " + srcFile.getCanonicalPath()
          + " to " + destFile.getCanonicalPath()
          + " due to failure in NativeIO#copyFileUnbuffered(). "
          + e.toString());
    }
    if (srcFile.length() != destFile.length()) {
      throw new IOException("Failed to copy full contents from '" + srcFile
          + "' to '" + destFile + "'");
    }
    if (preserveFileDate) {
      if (destFile.setLastModified(srcFile.lastModified()) == false) {

View on GitHub (pinned to 2add963021)

Solutions

Find and stop the process holding the file open (lsof <dest>) and retry
Verify the daemon user owns both the file and its directory (sticky-bit rule) and fix ownership
Check SELinux/audit logs for unlink denials and adjust the policy or contexts

Defensive patterns

Strategy: validation

Validate before calling

if (destFile.exists()) {
  File parent = destFile.getParentFile();
  boolean stickySafe = (parent == null) || !stickyBitSet(parent)
      || destFile.getCanonicalPath().startsWith(System.getProperty("user.home"));
  if (!destFile.delete()) throw new IOException("cannot delete " + destFile
      + "; held open or sticky-bit/ownership conflict");
}

Try / catch

try {
  Storage.nativeCopyFileUnbuffered(src, dst, true);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("cannot be deleted")) {
    // lsof <dst> to find the holder, stop it, retry once
  }
}

Prevention

Ensure only one process touches a storage file at a time
Run daemons and cleanup jobs as the owning user
On Windows, close all handles before checkpoint cleanup

When it happens

Trigger: Another process holds the destination open (Windows file locking, silly-rename on NFS); the directory has the sticky bit and the daemon user is not the owner; SELinux policy denies unlink; a virus scanner or backup tool has the file open.

Common situations: Two daemons or a monitoring tool touching the same storage files; NFS-mounted storage; hardened directories with the sticky bit; antivirus on Windows nodes.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a09e6a8198ca45e5. Report an issue: GitHub.