{"record":{"id":"e2d3809bf29a8cd9","repo":"apache/hadoop","slug":"failed-to-fully-delete-compressed-aliasmap-compre","errorCode":null,"errorMessage":"Failed to fully delete compressed aliasmap {compressedAliasMap}\nFailed to fully delete the aliasmap snapshot {aliasMapSnapshot}\n","messagePattern":"Failed to fully delete compressed aliasmap (.+?)\nFailed to fully delete the aliasmap snapshot (.+?)\n","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"warning","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java","lineNumber":270,"sourceCode":"        DataTransferThrottler throttler =\n            ImageServlet.getThrottlerForBootstrapStandby(conf);\n        TransferFsImage.copyFileToStream(response.getOutputStream(),\n            compressedAliasMap, fis, throttler);\n      }\n    } finally {\n      // cleanup the temporary snapshot and compressed files.\n      StringBuilder errMessage = new StringBuilder();\n      if (compressedAliasMap != null\n          && !FileUtil.fullyDelete(compressedAliasMap)) {\n        errMessage.append(\"Failed to fully delete compressed aliasmap \")\n            .append(compressedAliasMap.getAbsolutePath()).append(\"\\n\");\n      }\n      if (aliasMapSnapshot != null && !FileUtil.fullyDelete(aliasMapSnapshot)) {\n        errMessage.append(\"Failed to fully delete the aliasmap snapshot \")\n            .append(aliasMapSnapshot.getAbsolutePath()).append(\"\\n\");\n      }\n      if (errMessage.length() > 0) {\n        throw new IOException(errMessage.toString());\n      }\n    }\n  }\n\n  /**\n   * Create a new LevelDB store which is a snapshot copy of the original\n   * aliasmap.\n   *\n   * @param aliasMap original aliasmap.\n   * @return the {@link File} where the snapshot is created.\n   * @throws IOException\n   */\n  static File createSnapshot(InMemoryAliasMap aliasMap) throws IOException {\n    File originalAliasMapDir = new File(aliasMap.aliasMapURI);\n    String bpid = originalAliasMapDir.getName();\n    File snapshotDir =\n        new File(originalAliasMapDir.getParent(), SNAPSHOT_COPY_DIR);\n    File newLevelDBDir = new File(snapshotDir, bpid);","sourceCodeStart":252,"sourceCodeEnd":288,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java#L252-L288","documentation":"During bootstrap-transfer cleanup, InMemoryAliasMap concatenates an error message when the compressed aliasmap archive or the aliasmap snapshot directory cannot be fully deleted (FileUtil.fullyDelete returns false) and throws it as one IOException. The transfer itself may have succeeded; this failure is about removing temporary artifacts (aliasmap.tar.gz and the aliasmap_snapshot dir). Stranded temp artifacts can then break later snapshot attempts.","triggerScenarios":"After bootstrapTransfer work, cleanup calls FileUtil.fullyDelete on the compressed tarball and/or the snapshot dir; either delete fails (open handle, permissions on the tree, NFS silly-rename, read-only subtree) and the appended message is thrown.","commonSituations":"Journal/aliasmap dirs on NFS with held-open files; another process (backup, indexer) walking the snapshot dir during deletion; permission change between create and delete; running as a user that can create but not remove files created by a different uid (setuid/sticky-bit dir).","solutions":["Manually remove the leftovers while the HDFS service is stopped: rm -rf <leveldb.dir>/../aliasmap.tar.gz and <parent>/aliasmap_snapshot, then retry the bootstrap.","Fix ownership/permissions so the service user can delete everything under the aliasmap parent (needs write on each directory in the tree).","Stop third-party scanners/backup agents from holding files in the aliasmap tree (lsof +D to find holders).","Avoid NFS for the aliasmap directory; use local disk where delete semantics are reliable."],"exampleFix":"# before\n# IOException: Failed to fully delete compressed aliasmap /data/aliasmap.tar.gz\n#            Failed to fully delete the aliasmap snapshot /data/aliasmap_snapshot\n\n# after: stop services, clear stuck temp artifacts, fix perms, retry\nsudo -u hdfs hdfs --daemon stop namenode\nsudo rm -rf /data/aliasmap.tar.gz /data/aliasmap_snapshot\nsudo chown -R hdfs:hadoop /data\nsudo -u hdfs hdfs --daemon start namenode","handlingStrategy":"try-catch","validationCode":"File tar = new File(parent, \"aliasmap.tar.gz\");\nFile snap = new File(parent, \"aliasmap_snapshot\");\nif ((tar.exists() && !FileUtil.fullyDelete(tar))\n    || (snap.exists() && !FileUtil.fullyDelete(snap))) {\n  LOG.warn(\"Stale aliasmap temp artifacts present; clean manually before retry\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  InMemoryAliasMap.cleanup(...); // or the bootstrap flow whose cleanup may throw\n} catch (IOException e) {\n  if (e.getMessage().contains(\"Failed to fully delete\")) {\n    // transfer already succeeded: log, clean leftovers manually, continue\n    LOG.warn(\"Aliasmap temp cleanup failed; remove stale files\", e);\n  } else { throw e; }\n}","preventionTips":["Keep the aliasmap parent directory exclusively owned by the HDFS service user.","Host the aliasmap on local filesystem, not NFS (open handles block deletes).","Exclude aliasmap dirs from backup/indexer agents that can hold files open."],"tags":["hdfs","provided-storage","aliasmap","cleanup","file-delete"],"backgroundTag":"file-delete-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}