{"record":{"id":"6c0426b50bbe732b","repo":"apache/hadoop","slug":"storage-directory-for-location-and-block-pool-i","errorCode":null,"errorMessage":"Storage directory for location {} and block pool id {} does not exist","messagePattern":"Storage directory for location (.+?) and block pool id (.+?) does not exist","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java","lineNumber":164,"sourceCode":"   * @return\n   * @throws IOException\n   */\n  private StorageDirectory loadStorageDirectory(NamespaceInfo nsInfo,\n      StorageLocation location, StartupOption startOpt,\n      List<Callable<StorageDirectory>> callables, Configuration conf)\n          throws IOException {\n    StorageDirectory sd = new StorageDirectory(\n        nsInfo.getBlockPoolID(), null, true, location);\n    try {\n      StorageState curState = sd.analyzeStorage(startOpt, this, true);\n      // sd is locked but not opened\n      switch (curState) {\n      case NORMAL:\n        break;\n      case NON_EXISTENT:\n        LOG.info(\"Block pool storage directory for location {} and block pool\"\n            + \" id {} does not exist\", location, nsInfo.getBlockPoolID());\n        throw new IOException(\"Storage directory for location \" + location +\n            \" and block pool id \" + nsInfo.getBlockPoolID() +\n            \" does not exist\");\n      case NOT_FORMATTED: // format\n        LOG.info(\"Block pool storage directory for location {} and block pool\"\n                + \" id {} is not formatted. Formatting ...\", location,\n            nsInfo.getBlockPoolID());\n        format(sd, nsInfo);\n        break;\n      default:  // recovery part is common\n        sd.doRecover(curState);\n      }\n\n      // 2. Do transitions\n      // Each storage directory is treated individually.\n      // During startup some of them can upgrade or roll back\n      // while others could be up-to-date for the regular startup.\n      if (!doTransition(sd, nsInfo, startOpt, callables, conf)) {\n","sourceCodeStart":146,"sourceCodeEnd":182,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java#L146-L182","documentation":"While loading block pool storage, StorageDirectory.analyzeStorage returned NON_EXISTENT: the directory for this block pool at the given location is absent and the startup option is not FORMAT, so there is nothing to recover or transition and loadStorageDirectory throws.","triggerScenarios":"Starting the DN with -regular/-rollback/-upgrade against a data dir where <location>/<bpid> is missing; the BP dir was deleted or moved; the NN was re-formatted so a new bpid is expected while startOpt suppresses formatting.","commonSituations":"Post-reformat cleanup where stale BP dirs were removed but not the reverse; operators pruning BP directories by hand; rollback attempted without a preserved previous snapshot.","solutions":["Restart the DN so fresh registration formats the block pool directory for the current bpid","Restore the missing BP directory from backup if the blocks it held matter","Remove the bad entry from dfs.datanode.data.dir or correct the path"],"exampleFix":"# if the NN was re-formatted and data is disposable:\nrm -rf /dfs/dn/current/BP-*   # stale block pool dirs\nhdfs --daemon restart datanode  # re-registers and formats new BP dir","handlingStrategy":"validation","validationCode":"// Pre-flight before DN start when not formatting:\nfor (String dir : conf.getTrimmedStringCollection(DFS_DATANODE_DATA_DIR)) {\n  File bpDir = new File(dir, \"current/\" + expectedBpid);\n  if (!bpDir.exists() && startOpt != StartupOption.FORMAT) {\n    LOG.warn(\"Missing block pool dir {} - DN will fail to load it\", bpDir);\n  }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["After formatting/re-formatting the NN, remove stale BP dirs and let DNs re-register cleanly","Never delete BP directories you still need without a backup","Keep dfs.datanode.data.dir paths stable across restarts"],"tags":["hdfs","datanode","storage","block-pool","missing-directory"],"backgroundTag":"missing-storage-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}