{"record":{"id":"f5c4b6fd3b72f730","repo":"apache/hadoop","slug":"cannot-remove-directory-f5c4b6","errorCode":null,"errorMessage":"Cannot remove directory {}","messagePattern":"Cannot remove directory (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java","lineNumber":898,"sourceCode":"   * \n   * If the directory is not empty report an error; \n   * Otherwise remove the directory.\n   * \n   * @param detachDir detach directory\n   * @throws IOException if the directory is not empty or it can not be removed\n   */\n  private void cleanupDetachDir(File detachDir) throws IOException {\n    if (!DataNodeLayoutVersion.supports(\n        LayoutVersion.Feature.APPEND_RBW_DIR, layoutVersion) &&\n        detachDir.exists() && detachDir.isDirectory() ) {\n      \n        if (FileUtil.list(detachDir).length != 0 ) {\n          throw new IOException(\"Detached directory \" + detachDir +\n              \" is not empty. Please manually move each file under this \" +\n              \"directory to the finalized directory if the finalized \" +\n              \"directory tree does not have the file.\");\n        } else if (!detachDir.delete()) {\n          throw new IOException(\"Cannot remove directory \" + detachDir);\n        }\n    }\n  }\n  \n  /** \n   * Rolling back to a snapshot in previous directory by moving it to current\n   * directory.\n   * Rollback procedure:\n   * <br>\n   * If previous directory exists:\n   * <ol>\n   * <li> Rename current to removed.tmp </li>\n   * <li> Rename previous to current </li>\n   * <li> Remove removed.tmp </li>\n   * </ol>\n   * \n   * If previous directory does not exist and the current version supports\n   * federation, perform a simple rollback of layout version. This does not","sourceCodeStart":880,"sourceCodeEnd":916,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java#L880-L916","documentation":"In the same pre-APPEND_RBW_DIR rollback path, once the detach directory is empty it is removed with detachDir.delete(). A false return - permissions, a file appearing concurrently, or a filesystem error - triggers 'Cannot remove directory <dir>' and aborts the rollback.","triggerScenarios":"Rollback with an empty but undeletable detach directory: no write permission on the parent directory for the DN user, NFS/mount transient errors, or another process touching detach/ mid-rollback.","commonSituations":"Wrong ownership of the data dir tree; NFS-mounted DN storage with hiccups; racing processes creating files inside detach/.","solutions":["Fix ownership/permissions so the DN user can modify the parent directory, then retry rollback","Remove the empty detach directory manually as the DN user and retry","Do not use NFS for DN storage; check mount health (mount, dmesg) on that host"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// before rollback: DN user must be able to modify the parent of detach/\nPath detach = dir.resolve(\"detach\");\nif (Files.exists(detach)) {\n  Path parent = detach.getParent();\n  if (!Files.isWritable(parent)) {\n    throw new IOException(\"Cannot modify \" + parent + \" - fix ownership/permissions\");\n  }\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (String.valueOf(e.getMessage()).contains(\"Cannot remove directory\")) {\n    // permissions/mount issue: fix perms on the parent dir or remove detach/ manually, then retry\n  } else { throw e; }\n}","preventionTips":["Keep data dir trees owned by the DN user throughout, including legacy detach/ dirs","Avoid NFS for datanode storage; check mount health before rollback operations","Remove empty legacy detach/ directories during routine maintenance so rollbacks never hit them"],"tags":["hadoop","hdfs","datanode","storage","rollback","permissions","filesystem"],"backgroundTag":"directory-delete-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}