apache/hadoop · warning · IOException

Failed to fully delete compressed aliasmap {compressedAliasM

Error message

Failed to fully delete compressed aliasmap {compressedAliasMap}
Failed to fully delete the aliasmap snapshot {aliasMapSnapshot}

What it means

During bootstrap-transfer cleanup, InMemoryAliasMap concatenates an error message when the compressed aliasmap archive or the aliasmap snapshot directory cannot be fully deleted (FileUtil.fullyDelete returns false) and throws it as one IOException. The transfer itself may have succeeded; this failure is about removing temporary artifacts (aliasmap.tar.gz and the aliasmap_snapshot dir). Stranded temp artifacts can then break later snapshot attempts.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java:270

        DataTransferThrottler throttler =
            ImageServlet.getThrottlerForBootstrapStandby(conf);
        TransferFsImage.copyFileToStream(response.getOutputStream(),
            compressedAliasMap, fis, throttler);
      }
    } finally {
      // cleanup the temporary snapshot and compressed files.
      StringBuilder errMessage = new StringBuilder();
      if (compressedAliasMap != null
          && !FileUtil.fullyDelete(compressedAliasMap)) {
        errMessage.append("Failed to fully delete compressed aliasmap ")
            .append(compressedAliasMap.getAbsolutePath()).append("\n");
      }
      if (aliasMapSnapshot != null && !FileUtil.fullyDelete(aliasMapSnapshot)) {
        errMessage.append("Failed to fully delete the aliasmap snapshot ")
            .append(aliasMapSnapshot.getAbsolutePath()).append("\n");
      }
      if (errMessage.length() > 0) {
        throw new IOException(errMessage.toString());
      }
    }
  }

  /**
   * Create a new LevelDB store which is a snapshot copy of the original
   * aliasmap.
   *
   * @param aliasMap original aliasmap.
   * @return the {@link File} where the snapshot is created.
   * @throws IOException
   */
  static File createSnapshot(InMemoryAliasMap aliasMap) throws IOException {
    File originalAliasMapDir = new File(aliasMap.aliasMapURI);
    String bpid = originalAliasMapDir.getName();
    File snapshotDir =
        new File(originalAliasMapDir.getParent(), SNAPSHOT_COPY_DIR);
    File newLevelDBDir = new File(snapshotDir, bpid);

View on GitHub (pinned to 2add963021)

Solutions

Manually remove the leftovers while the HDFS service is stopped: rm -rf <leveldb.dir>/../aliasmap.tar.gz and <parent>/aliasmap_snapshot, then retry the bootstrap.
Fix ownership/permissions so the service user can delete everything under the aliasmap parent (needs write on each directory in the tree).
Stop third-party scanners/backup agents from holding files in the aliasmap tree (lsof +D to find holders).
Avoid NFS for the aliasmap directory; use local disk where delete semantics are reliable.

Example fix

# before
# IOException: Failed to fully delete compressed aliasmap /data/aliasmap.tar.gz
#            Failed to fully delete the aliasmap snapshot /data/aliasmap_snapshot

# after: stop services, clear stuck temp artifacts, fix perms, retry
sudo -u hdfs hdfs --daemon stop namenode
sudo rm -rf /data/aliasmap.tar.gz /data/aliasmap_snapshot
sudo chown -R hdfs:hadoop /data
sudo -u hdfs hdfs --daemon start namenode

Defensive patterns

Strategy: try-catch

Validate before calling

File tar = new File(parent, "aliasmap.tar.gz");
File snap = new File(parent, "aliasmap_snapshot");
if ((tar.exists() && !FileUtil.fullyDelete(tar))
    || (snap.exists() && !FileUtil.fullyDelete(snap))) {
  LOG.warn("Stale aliasmap temp artifacts present; clean manually before retry");
}

Try / catch

try {
  InMemoryAliasMap.cleanup(...); // or the bootstrap flow whose cleanup may throw
} catch (IOException e) {
  if (e.getMessage().contains("Failed to fully delete")) {
    // transfer already succeeded: log, clean leftovers manually, continue
    LOG.warn("Aliasmap temp cleanup failed; remove stale files", e);
  } else { throw e; }
}

Prevention

Keep the aliasmap parent directory exclusively owned by the HDFS service user.
Host the aliasmap on local filesystem, not NFS (open handles block deletes).
Exclude aliasmap dirs from backup/indexer agents that can hold files open.

When it happens

Trigger: After bootstrapTransfer work, cleanup calls FileUtil.fullyDelete on the compressed tarball and/or the snapshot dir; either delete fails (open handle, permissions on the tree, NFS silly-rename, read-only subtree) and the appended message is thrown.

Common situations: Journal/aliasmap dirs on NFS with held-open files; another process (backup, indexer) walking the snapshot dir during deletion; permission change between create and delete; running as a user that can create but not remove files created by a different uid (setuid/sticky-bit dir).

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e2d3809bf29a8cd9. Report an issue: GitHub.