{"record":{"id":"e1997569cc99024b","repo":"apache/hadoop","slug":"missing-directory","errorCode":null,"errorMessage":"Missing directory {}","messagePattern":"Missing directory (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java","lineNumber":158,"sourceCode":"    minimumRedundantVolumes = conf.getInt(\n        DFSConfigKeys.DFS_NAMENODE_CHECKED_VOLUMES_MINIMUM_KEY,\n        DFSConfigKeys.DFS_NAMENODE_CHECKED_VOLUMES_MINIMUM_DEFAULT);\n  }\n\n  /**\n   * Add the volume of the passed-in directory to the list of volumes to check.\n   * If <code>required</code> is true, and this volume is already present, but\n   * is marked redundant, it will be marked required. If the volume is already\n   * present but marked required then this method is a no-op.\n   * \n   * @param directoryToCheck\n   *          The directory whose volume will be checked for available space.\n   */\n  private void addDirToCheck(URI directoryToCheck, boolean required)\n      throws IOException {\n    File dir = new File(directoryToCheck.getPath());\n    if (!dir.exists()) {\n      throw new IOException(\"Missing directory \"+dir.getAbsolutePath());\n    }\n    \n    CheckedVolume newVolume = new CheckedVolume(dir, required);\n    CheckedVolume volume = volumes.get(newVolume.getVolume());\n    if (volume == null || !volume.isRequired()) {\n      volumes.put(newVolume.getVolume(), newVolume);\n    }\n  }\n\n  /**\n   * Return true if disk space is available on at least one of the configured\n   * redundant volumes, and all of the configured required volumes.\n   * \n   * @return True if the configured amount of disk space is available on at\n   *         least one redundant volume and all of the required volumes, false\n   *         otherwise.\n   */\n  public boolean hasAvailableDiskSpace() {","sourceCodeStart":140,"sourceCodeEnd":176,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java#L140-L176","documentation":"NameNodeResourceChecker's constructor registers every LOCAL edits dir (file:// entries of the edits-dir config) plus every URI in dfs.namenode.resource.checked.volumes (all marked required) as CheckedVolumes; addDirToCheck does File.exists() on each path and throws IOException('Missing directory <absolutePath>') if it does not exist. This runs during NameNode/FSNamesystem startup, so a missing dir aborts NameNode startup.","triggerScenarios":"A configured local edits directory or a dfs.namenode.resource.checked.volumes path absent on disk at NameNode start - mount not mounted, path typo, directory deleted, or a fresh host where the data dir was never created.","commonSituations":"Disk or NFS mount missing after a host reboot; typo'd checked-volume URI; someone deleted an edits dir during cleanup; container/k8s hostPath or volume mounts not created before the NN container starts.","solutions":["Create the directory with correct ownership (`mkdir -p <path>` and `chown hdfs:hadoop <path>`) and restart the NameNode","Fix the URI typo in dfs.namenode.edits.dir or dfs.namenode.resource.checked.volumes","If the volume may legitimately be absent, remove it from dfs.namenode.resource.checked.volumes or lower dfs.namenode.resource.checked.volumes.minimum","Ensure mount units (fstab/systemd) are active before the NameNode service starts; pre-create dirs in provisioning so NN start is idempotent"],"exampleFix":"# before - NameNode fails at startup: IOException Missing directory /data2/dfs/edits\nls /data2   # mount missing\n\n# after\nmkdir -p /data2/dfs/edits && chown hdfs:hadoop /data2/dfs/edits\nhdfs --daemon start namenode","handlingStrategy":"validation","validationCode":"# Preflight before NameNode start: every configured local edits/checked dir must exist\nfor u in $(hdfs getconf -nameDirsWithEdits 2>/dev/null | tr ',' ' '); do\n  p=${u#file://}\n  [ -d \"$p\" ] || { mkdir -p \"$p\" && chown hdfs:hadoop \"$p\"; }\ndone\nfor v in $(hdfs getconf -confKey dfs.namenode.resource.checked.volumes | tr ',' ' '); do\n  [ -d \"$v\" ] || { echo \"FATAL: checked volume $v missing - mount?\"; exit 1; }\ndone","typeGuard":"import java.nio.file.*;\nstatic boolean dirReady(String p) {\n  Path d = Paths.get(p);\n  return Files.isDirectory(d) && Files.isWritable(d);\n}","tryCatchPattern":"catch (IOException e) {\n  if (e.getMessage().startsWith(\"Missing directory\")) {\n    // create or mount the directory, verify ownership, then retry NN start\n  }\n}","preventionTips":["Order systemd units so mounts are active before the NameNode unit","Pre-create edits dirs and checked volumes in provisioning (Ansible/Helm)","Never delete directories under the edits-dir config during cleanup; exclude them from tmp cleaners","Monitor mounts - an unmounted data disk later surfaces as missing-dir or disk failures"],"tags":["hdfs","name-node","startup","edits-dir","filesystem","mount","resource-checker"],"backgroundTag":"missing-required-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}