{"record":{"id":"29f366ffdde992cc","repo":"apache/hadoop","slug":"checkpoint-directory-does-not-exist-or-is-not-acce-29f366","errorCode":null,"errorMessage":"checkpoint directory does not exist or is not accessible.","messagePattern":"checkpoint directory does not exist or is not accessible\\.","errorType":"exception","errorClass":"InconsistentFSStateException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java","lineNumber":1004,"sourceCode":"        }\n        if(!isAccessible)\n          throw new InconsistentFSStateException(sd.getRoot(),\n              \"cannot access checkpoint directory.\");\n        \n        if (format) {\n          // Don't confirm, since this is just the secondary namenode.\n          LOG.info(\"Formatting storage directory \" + sd);\n          sd.clearDirectory();\n        }\n        \n        StorageState curState;\n        try {\n          curState = sd.analyzeStorage(HdfsServerConstants.StartupOption.REGULAR, storage);\n          // sd is locked but not opened\n          switch(curState) {\n          case NON_EXISTENT:\n            // fail if any of the configured checkpoint dirs are inaccessible \n            throw new InconsistentFSStateException(sd.getRoot(),\n                  \"checkpoint directory does not exist or is not accessible.\");\n          case NOT_FORMATTED:\n            break;  // it's ok since initially there is no current and VERSION\n          case NORMAL:\n            // Read the VERSION file. This verifies that:\n            // (a) the VERSION file for each of the directories is the same,\n            // and (b) when we connect to a NN, we can verify that the remote\n            // node matches the same namespace that we ran on previously.\n            storage.readProperties(sd);\n            break;\n          default:  // recovery is possible\n            sd.doRecover(curState);\n          }\n        } catch (IOException ioe) {\n          sd.unlock();\n          throw ioe;\n        }\n      }","sourceCodeStart":986,"sourceCodeEnd":1022,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java#L986-L1022","documentation":"SecondaryNameNode throws InconsistentFSStateException when StorageDirectory.analyzeStorage(StartupOption.REGULAR) returns NON_EXISTENT: after the (non-throwing) mkdirs() attempt the checkpoint directory still does not exist, so it is either uncreatable or inaccessible. This is the common 'checkpoint dir missing' failure that stops the 2NN at startup.","triggerScenarios":"analyzeStorage() finds neither a usable directory state (NORMAL/NOT_FORMATTED/recoverable) nor an existing root — typically because mkdirs() silently returned false (read-only mount, missing parent, parent path occupied by a regular file) or permissions prevent creation.","commonSituations":"dfs.namenode.checkpoint.dir points to a path on an NFS mount mounted read-only; the directory was deleted or never created on a new host; a file exists where the directory should be; the 2NN user lacks create permission on the parent.","solutions":["Create the directory manually with the right ownership: mkdir -p <dir> && chown <2nn-user> <dir>, then restart the 2NN","Check the parent chain of the configured path is writable by the 2NN user and that no regular file shadows any path component","Verify the mount options (rw, not ro) of the filesystem holding the checkpoint dir","Correct a mistyped dfs.namenode.checkpoint.dir value in hdfs-site.xml"],"exampleFix":"# before\n$ hdfs secondarynamenode  # InconsistentFSStateException: checkpoint directory does not exist or is not accessible.\n# after\n$ sudo mkdir -p /data/2nn/checkpoint\n$ sudo chown hdfs:hadoop /data/2nn/checkpoint\n$ hdfs secondarynamenode","handlingStrategy":"validation","validationCode":"for (String dir : conf.getTrimmedStrings(DFSConfigKeys.DFS_NAMENODE_CHECKPOINT_DIR_KEY)) {\n  File f = new File(dir);\n  File parent = f.getParentFile();\n  boolean creatable = (f.exists() && f.isDirectory())\n      || (parent != null && parent.canWrite() && parent.isDirectory());\n  if (!creatable) {\n    throw new IOException(\"Checkpoint dir missing and not creatable: \" + dir);\n  }\n}","typeGuard":null,"tryCatchPattern":"catch (InconsistentFSStateException e) {\n  if (e.getMessage().contains(\"does not exist or is not accessible\")) {\n    // create/repair the directory out-of-band, then restart the 2NN once\n  }\n}","preventionTips":["Create checkpoint directories during node provisioning and verify mounts are rw before starting the 2NN","Add a pre-flight check script (df/mount/ls) to 2NN service start (systemd ExecStartPre)","Monitor for silently failing mkdirs by checking directory existence after any storage-related change"],"tags":["hdfs","secondary-namenode","checkpoint","missing-directory","storage","startup"],"backgroundTag":"directory-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}