{"record":{"id":"9e153838375ff47e","repo":"apache/hadoop","slug":"storage-directory-with-location-does-not-exist","errorCode":null,"errorMessage":"Storage directory with location {} does not exist","messagePattern":"Storage directory with location (.+?) does not exist","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java","lineNumber":281,"sourceCode":"        }\n        storage.addStorageDir(sd);\n      }\n    }\n  }\n\n  private StorageDirectory loadStorageDirectory(DataNode datanode,\n      NamespaceInfo nsInfo, StorageLocation location, StartupOption startOpt,\n      List<Callable<StorageDirectory>> callables) throws IOException {\n    StorageDirectory sd = new StorageDirectory(null, false, location);\n    try {\n      StorageState curState = sd.analyzeStorage(startOpt, this, true);\n      // sd is locked but not opened\n      switch (curState) {\n      case NORMAL:\n        break;\n      case NON_EXISTENT:\n        LOG.info(\"Storage directory with location {} does not exist\", location);\n        throw new IOException(\"Storage directory with location \" + location\n            + \" does not exist\");\n      case NOT_FORMATTED: // format\n        LOG.info(\"Storage directory with location {} is not formatted for \"\n            + \"namespace {}. Formatting...\", location, nsInfo.getNamespaceID());\n        format(sd, nsInfo, datanode.getDatanodeUuid(), datanode.getConf());\n        break;\n      default:  // recovery part is common\n        sd.doRecover(curState);\n      }\n\n      // 2. Do transitions\n      // Each storage directory is treated individually.\n      // During startup some of them can upgrade or roll back\n      // while others could be up-to-date for the regular startup.\n      if (!doTransition(sd, nsInfo, startOpt, callables, datanode.getConf())) {\n\n        // 3. Update successfully loaded storage.\n        setServiceLayoutVersion(getServiceLayoutVersion());","sourceCodeStart":263,"sourceCodeEnd":299,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java#L263-L299","documentation":"During DataStorage.loadStorageDirectory, StorageDirectory.analyzeStorage reports NON_EXISTENT when the configured location does not exist as a directory and is not being created on this path (non-FORMAT startup). The DN logs the location and aborts loading that directory with this IOException.","triggerScenarios":"Configuring a dfs.datanode.data.dir entry (or hot-swapping a volume in) whose path does not exist and cannot be created: missing parent directory, typo, no mkdir permission, or the underlying mount/device is gone.","commonSituations":"Typos in data.dir paths; failed or unmounted disk; mount points not yet created when the DN starts; container/k8s volume mounts missing; parent directory read-only for the DN user.","solutions":["Verify the path exists and the DN user can create/write it: sudo -u hdfs mkdir -p <dir> && sudo -u hdfs touch <dir>/.write_test","Fix typos in dfs.datanode.data.dir","Restore the missing mount (fstab / k8s volume) and restart the DN","If the disk is dead, remove that entry from dfs.datanode.data.dir before restarting"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"for (String dir : dirs) {\n  Path p = Paths.get(dir);\n  Files.createDirectories(p);              // fails fast with clear reason\n  if (!Files.isWritable(p)) throw new IOException(\"Not writable: \" + p);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pre-create data dirs and chown them to the DN user in host provisioning","Smoke-test existence + writability of every dfs.datanode.data.dir entry before starting/reconfiguring the DN","Fstab/systemd ordering: ensure mounts exist before the datanode service starts"],"tags":["hadoop","hdfs","datanode","storage","data-dir","filesystem"],"backgroundTag":"storage-directory-missing","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}