{"record":{"id":"5d167741026ee14d","repo":"apache/hadoop","slug":"directory-dir-is-in-an-inconsistent-state-stora","errorCode":null,"errorMessage":"Directory {dir} is in an inconsistent state: storage directory does not exist or is not accessible.","messagePattern":"Directory (.+?) is in an inconsistent state: storage directory does not exist or is not accessible\\.","errorType":"exception","errorClass":"org.apache.hadoop.hdfs.server.common.InconsistentFSStateException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java","lineNumber":393,"sourceCode":"    // sure that we properly examine their state, but we make sure we don't\n    // mutate the shared dir below in the actual loop.\n    for (Iterator<StorageDirectory> it = \n                      storage.dirIterator(); it.hasNext();) {\n      StorageDirectory sd = it.next();\n      StorageState curState;\n      if (startOpt == StartupOption.METADATAVERSION) {\n        /* All we need is the layout version. */\n        storage.readProperties(sd);\n        return true;\n      }\n\n      try {\n        curState = sd.analyzeStorage(startOpt, storage);\n        // sd is locked but not opened\n        switch(curState) {\n        case NON_EXISTENT:\n          // name-node fails if any of the configured storage dirs are missing\n          throw new InconsistentFSStateException(sd.getRoot(),\n                      \"storage directory does not exist or is not accessible.\");\n        case NOT_FORMATTED:\n          break;\n        case NORMAL:\n          break;\n        default:  // recovery is possible\n          sd.doRecover(curState);\n        }\n        if (curState != StorageState.NOT_FORMATTED \n            && startOpt != StartupOption.ROLLBACK) {\n          // read and verify consistency with other directories\n          storage.readProperties(sd, startOpt);\n          isFormatted = true;\n        }\n        if (startOpt == StartupOption.IMPORT && isFormatted)\n          // import of a checkpoint is allowed only into empty image directories\n          throw new IOException(\"Cannot import image from a checkpoint. \" \n              + \" NameNode already contains an image in \" + sd.getRoot());","sourceCodeStart":375,"sourceCodeEnd":411,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java#L375-L411","documentation":"In recoverStorageDirs(), StorageDirectory.analyzeStorage() returned NON_EXISTENT for a configured storage directory: the directory does not exist or is not accessible. Unlike the DataNode, the NameNode never creates directories on demand — any configured-but-missing directory is fatal (InconsistentFSStateException).","triggerScenarios":"A dfs.namenode.name.dir / edits.dir entry deleted, never created, on an unmounted volume, or unreadable due to permissions; path typo in hdfs-site.xml; NFS or aggregate volume unavailable at boot.","commonSituations":"Disk replaced or not mounted after reboot; directory removed by cleanup jobs; container/pod started without the hostPath volume; ownership/ACL changes after a user rename.","solutions":["Inspect the directory named in the message: ls -ld <dir>, check the mount (df -h) and ownership.","Create it and fix ownership/perms: mkdir -p <dir> && chown hdfs:hdfs <dir> && chmod 700 <dir>.","Correct the config if the path is a typo (hdfs getconf -confKey dfs.namenode.name.dir).","If the volume is permanently gone, restore its metadata from a sibling dir or backup — or remove the entry from config only if other dirs hold copies of the metadata."],"exampleFix":"# before\nls -ld /dfs/nn1   # No such file or directory\n\n# after\nmkdir -p /dfs/nn1\nchown hdfs:hdfs /dfs/nn1\nchmod 700 /dfs/nn1\nhdfs --daemon start namenode","handlingStrategy":"validation","validationCode":"for (URI u : allConfiguredDirs) {\n  Path p = Paths.get(u);\n  if (!Files.isDirectory(p)) {\n    fail(u + \" does not exist\");\n  } else if (!Files.isReadable(p) || !Files.isWritable(p)) {\n    fail(u + \" not accessible\");\n  }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Preflight directory existence and permissions before every NameNode start.","Monitor mounts for name dirs (NFS especially).","Keep directory provisioning and hdfs-site.xml in the same playbook."],"tags":["hdfs","namenode","storage-directory","permissions","startup"],"backgroundTag":"missing-storage-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}