{"record":{"id":"bde34221cd011bcd","repo":"apache/hadoop","slug":"exception-while-processing-storagedirectory-sd","errorCode":null,"errorMessage":"Exception while processing StorageDirectory \" + sd.getRoot()","messagePattern":"Exception while processing StorageDirectory \" \\+ sd\\.getRoot\\(\\)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java","lineNumber":433,"sourceCode":"\n  /**\n   * Return the list of locations being used for a specific purpose.\n   * i.e. Image or edit log storage.\n   *\n   * @param dirType Purpose of locations requested.\n   * @throws IOException\n   */\n  Collection<URI> getDirectories(NameNodeDirType dirType)\n      throws IOException {\n    ArrayList<URI> list = new ArrayList<>();\n    Iterator<StorageDirectory> it = (dirType == null) ? dirIterator() :\n                                    dirIterator(dirType);\n    for ( ; it.hasNext();) {\n      StorageDirectory sd = it.next();\n      try {\n        list.add(Util.fileAsURI(sd.getRoot()));\n      } catch (IOException e) {\n        throw new IOException(\"Exception while processing \" +\n            \"StorageDirectory \" + sd.getRoot(), e);\n      }\n    }\n    return list;\n  }\n  \n  /**\n   * Determine the last transaction ID noted in this storage directory.\n   * This txid is stored in a special seen_txid file since it might not\n   * correspond to the latest image or edit log. For example, an image-only\n   * directory will have this txid incremented when edits logs roll, even\n   * though the edits logs are in a different directory.\n   *\n   * @param sd StorageDirectory to check\n   * @return If file exists and can be read, last recorded txid. If not, 0L.\n   * @throws IOException On errors processing file pointed to by sd\n   */\n  static long readTransactionIdFile(StorageDirectory sd) throws IOException {","sourceCodeStart":415,"sourceCodeEnd":451,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java#L415-L451","documentation":"NNStorage.getDirectories(dirType) walks each StorageDirectory of the requested type and converts its root with Util.fileAsURI; if that conversion fails - most commonly a relative local path, which fileAsURI refuses - the failure is wrapped with the offending directory root in the message. It typically surfaces while collecting IMAGE or EDITS directory lists for checkpoint or transfer setup.","triggerScenarios":"dfs.namenode.name.dir or dfs.namenode.edits.dir contains a relative path such as 'dfs/name' (no leading '/', no file: scheme), so Util.fileAsURI(sd.getRoot()) throws when the NN builds its storage or the checkpoint lists directories.","commonSituations":"Configs copied from unit tests that use relative dirs; ${var} substitution resolving to a relative path; NN working directory changed by a new init script, turning previously-absolute-enough paths relative.","solutions":["Make every name/edits directory absolute: /data/nn or file:///data/nn.","Check ${...} expansion in the XML for the exact directory named in the message.","After fixing, verify with 'hdfs getconf -nameDirs' and 'hdfs getconf -backupNodes'... or simply re-run getconf -confKey to confirm the effective value."],"exampleFix":"// before - relative path\n<property>\n  <name>dfs.namenode.name.dir</name>\n  <value>dfs/name</value>\n</property>\n\n// after - absolute path\n<property>\n  <name>dfs.namenode.name.dir</name>\n  <value>file:///data/dfs/name</value>\n</property>","handlingStrategy":"validation","validationCode":"// Guard every configured storage dir is resolvable to an absolute file URI\nfor (String loc : conf.getTrimmedStrings(\"dfs.namenode.name.dir\")) {\n  File f = new File(loc);\n  boolean absolute = f.isAbsolute() || (loc.startsWith(\"file:\"));\n  if (!absolute) {\n    throw new IllegalArgumentException(\"Relative storage dir rejected: \" + loc\n        + \" - use /abs/path or file:///abs/path\");\n  }\n}","typeGuard":"static boolean isAbsoluteStorageDir(String location) {\n  if (location == null) return false;\n  if (location.startsWith(\"file:\")) return URI.create(location).getPath().startsWith(\"/\");\n  return new File(location).isAbsolute();\n}","tryCatchPattern":null,"preventionTips":["Ban relative paths in name/edits dir configs via config linting.","After template substitution, re-check the effective values with hdfs getconf.","Pin the NN working directory in systemd so even legacy relative paths resolve deterministically."],"tags":["hdfs","configuration","path","storage"],"backgroundTag":"invalid-storage-directory-path","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}