{"record":{"id":"879818ce1a8376b8","repo":"apache/hadoop","slug":"detached-directory-is-not-empty-please-manuall-879818","errorCode":null,"errorMessage":"Detached directory {} is not empty. Please manually move each file under this directory to the finalized directory if the finalized directory tree does not have the file.","messagePattern":"Detached directory (.+?) is not empty\\. Please manually move each file under this directory to the finalized directory if the finalized directory tree does not have the file\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java","lineNumber":893,"sourceCode":"    writeProperties(sd);\n  }\n\n  /**\n   * Cleanup the detachDir. \n   * \n   * If the directory is not empty report an error; \n   * Otherwise remove the directory.\n   * \n   * @param detachDir detach directory\n   * @throws IOException if the directory is not empty or it can not be removed\n   */\n  private void cleanupDetachDir(File detachDir) throws IOException {\n    if (!DataNodeLayoutVersion.supports(\n        LayoutVersion.Feature.APPEND_RBW_DIR, layoutVersion) &&\n        detachDir.exists() && detachDir.isDirectory() ) {\n      \n        if (FileUtil.list(detachDir).length != 0 ) {\n          throw new IOException(\"Detached directory \" + detachDir +\n              \" is not empty. Please manually move each file under this \" +\n              \"directory to the finalized directory if the finalized \" +\n              \"directory tree does not have the file.\");\n        } else if (!detachDir.delete()) {\n          throw new IOException(\"Cannot remove directory \" + detachDir);\n        }\n    }\n  }\n  \n  /** \n   * Rolling back to a snapshot in previous directory by moving it to current\n   * directory.\n   * Rollback procedure:\n   * <br>\n   * If previous directory exists:\n   * <ol>\n   * <li> Rename current to removed.tmp </li>\n   * <li> Rename previous to current </li>","sourceCodeStart":875,"sourceCodeEnd":911,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java#L875-L911","documentation":"During rollback of layouts older than APPEND_RBW_DIR, cleanupDetachDir inspects the legacy detach directory. If it still contains files (leftovers from the old append/copy-on-write mechanics), the DN cannot tell whether those blocks also exist in finalized, so it refuses the rollback and asks the operator to reconcile the files manually.","triggerScenarios":"Running 'hdfs datanode -rollback' on a storage dir whose layout predates APPEND_RBW_DIR while <dir>/detach still holds files from earlier append operations.","commonSituations":"Old clusters exercising rollback after an upgrade that had used appends; upgrade rehearsals on historical dirs; restored legacy directories containing detach leftovers.","solutions":["Follow the message: for each file under detach/, move it into the finalized tree only if finalized does not already contain that block","Once detach/ is empty, retry the rollback","Back up detach/ before doing any manual moves"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// before 'hdfs datanode -rollback' on legacy layouts\nPath detach = dir.resolve(\"detach\");\nif (Files.isDirectory(detach)\n    && Files.list(detach).findAny().isPresent()) {\n  throw new IOException(\"detach/ not empty under \" + dir\n      + \" - reconcile files into finalized/ first (skip blocks finalized already has)\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Before rollback rehearsals on old clusters, audit every data dir for leftover detach/ contents","Move detach files into finalized only when finalized lacks that block, then empty detach/","Snapshot/backup detach/ before manual reconciliation"],"tags":["hadoop","hdfs","datanode","storage","rollback","detach","legacy"],"backgroundTag":"stale-directory-not-empty","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}