{"record":{"id":"d53621f5d4e43f0a","repo":"apache/hadoop","slug":"detached-directory-is-not-empty-please-manuall","errorCode":null,"errorMessage":"Detached directory {} is not empty. Please manually move each file under this directory to the finalized directory if the finalized directory tree does not have the file.","messagePattern":"Detached directory (.+?) is not empty\\. Please manually move each file under this directory to the finalized directory if the finalized directory tree does not have the file\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java","lineNumber":536,"sourceCode":"    LOG.info(\"Upgrade of {} is complete\", name);\n  }\n\n  /**\n   * Cleanup the detachDir.\n   * \n   * If the directory is not empty report an error; Otherwise remove the\n   * directory.\n   * \n   * @param detachDir detach directory\n   * @throws IOException if the directory is not empty or it can not be removed\n   */\n  private void cleanupDetachDir(File detachDir) throws IOException {\n    if (!DataNodeLayoutVersion.supports(\n        LayoutVersion.Feature.APPEND_RBW_DIR, layoutVersion)\n        && detachDir.exists() && detachDir.isDirectory()) {\n\n      if (FileUtil.list(detachDir).length != 0) {\n        throw new IOException(\"Detached directory \" + detachDir\n            + \" is not empty. Please manually move each file under this \"\n            + \"directory to the finalized directory if the finalized \"\n            + \"directory tree does not have the file.\");\n      } else if (!detachDir.delete()) {\n        throw new IOException(\"Cannot remove directory \" + detachDir);\n      }\n    }\n  }\n\n  /**\n   * Restore all files from the trash directory to their corresponding\n   * locations under current/\n   */\n  private int restoreBlockFilesFromTrash(File trashRoot)\n      throws  IOException {\n    int filesRestored = 0;\n    File[] children = trashRoot.exists() ? trashRoot.listFiles() : null;\n    if (children == null) {","sourceCodeStart":518,"sourceCodeEnd":554,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java#L518-L554","documentation":"Legacy upgrade path: on layout versions older than APPEND_RBW_DIR, the upgrade requires the old detached/ directory (left by pre-0.21 append hard-link 'detach') to be empty. It still contains files, so cleanupDetachDir aborts the upgrade rather than silently dropping block files.","triggerScenarios":"Upgrading DataNode storage from a very old release that used the detach mechanism, with files still sitting in <bp>/detached/.","commonSituations":"Clusters upgraded across many releases in one hop; storage archives restored from ancient snapshots.","solutions":["Move each file from detached/ into the finalized/ tree only where finalized does not already have it (exactly as the message instructs)","Delete detached files that already exist with the same or larger size under finalized/","Re-run the DN upgrade once detached/ is empty"],"exampleFix":"# for f in /dfs/dn/current/BP-*/detached/*; do\n#   tgt=/dfs/dn/current/BP-*/current/finalized/...  # matching subpath\n#   [ -e \"$tgt\" ] || mv \"$f\" \"$tgt\"\n# done\nhdfs --daemon start datanode -upgrade","handlingStrategy":"validation","validationCode":"// Pre-upgrade check on legacy layouts:\nFile detach = new File(bpCurrentDir, \"detached\");\nif (detach.isDirectory() && FileUtil.list(detach).length > 0) {\n  throw new IOException(\"detached/ not empty; reconcile with finalized/ before upgrade\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Before hopping an old DN across many releases, audit for legacy detached/ directories","Reconcile detached files with finalized copies, then delete detached/ so the upgrade can proceed"],"tags":["hdfs","datanode","storage","upgrade","legacy"],"backgroundTag":"orphan-storage-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}