apache/hadoop · error · IllegalArgumentException

Required edits directory {} not found: dfs.namenode.edits.di

Error message

Required edits directory {} not found: dfs.namenode.edits.dir={}; dfs.namenode.edits.dir.required={}; dfs.namenode.shared.edits.dir={}

What it means

During FSNamesystem configuration validation, every URI in dfs.namenode.edits.dir.required (minus the default placeholder) must also appear in dfs.namenode.edits.dir or dfs.namenode.shared.edits.dir. A required dir configured nowhere else aborts NN startup with this IllegalArgumentException listing all three settings. Comparison is exact, so scheme/spelling differences count as missing.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:803

        FSNamesystem.getNamespaceDirs(conf);
    final Collection<URI> editsDirs =
        FSNamesystem.getNamespaceEditsDirs(conf);
    final Collection<URI> requiredEditsDirs =
        FSNamesystem.getRequiredNamespaceEditsDirs(conf);
    final Collection<URI> sharedEditsDirs =
        FSNamesystem.getSharedEditsDirs(conf);

    for (URI u : requiredEditsDirs) {
      if (u.toString().compareTo(
              DFSConfigKeys.DFS_NAMENODE_EDITS_DIR_DEFAULT) == 0) {
        continue;
      }

      // Each required directory must also be in editsDirs or in
      // sharedEditsDirs.
      if (!editsDirs.contains(u) &&
          !sharedEditsDirs.contains(u)) {
        throw new IllegalArgumentException("Required edits directory " + u
            + " not found: "
            + DFSConfigKeys.DFS_NAMENODE_EDITS_DIR_KEY + "=" + editsDirs + "; "
            + DFSConfigKeys.DFS_NAMENODE_EDITS_DIR_REQUIRED_KEY
            + "=" + requiredEditsDirs + "; "
            + DFSConfigKeys.DFS_NAMENODE_SHARED_EDITS_DIR_KEY
            + "=" + sharedEditsDirs);
      }
    }

    if (namespaceDirs.size() == 1) {
      LOG.warn("Only one image storage directory ("
          + DFS_NAMENODE_NAME_DIR_KEY + ") configured. Beware of data loss"
          + " due to lack of redundant storage directories!");
    }
    if (editsDirs.size() == 1) {
      LOG.warn("Only one namespace edits storage directory ("
          + DFS_NAMENODE_EDITS_DIR_KEY + ") configured. Beware of data loss"
          + " due to lack of redundant storage directories!");

View on GitHub (pinned to 2add963021)

Solutions

1. Add the required URI verbatim to dfs.namenode.edits.dir (or to dfs.namenode.shared.edits.dir if it is a shared journal)
2. Normalize URI spelling: identical scheme, authority and path in both properties
3. Drop the entry from dfs.namenode.edits.dir.required if it no longer needs to be mandatory
4. Restart the NameNode and confirm the validation passes

Example fix

<!-- before -->
<property><name>dfs.namenode.edits.dir.required</name><value>file:///dfs/edits</value></property>
<property><name>dfs.namenode.edits.dir</name><value>file:///dfs/name</value></property>
<!-- after: the required dir is listed in edits.dir, spelled identically -->
<property><name>dfs.namenode.edits.dir</name><value>file:///dfs/edits,file:///dfs/name</value></property>

Collection<URI> edits = FSNamesystem.getEditsDirs(conf);
for (URI u : FSNamesystem.getRequiredEditsDirs(conf)) {
  if (u.toString().equals(DFSConfigKeys.DFS_NAMENODE_EDITS_DIR_DEFAULT)) continue;
  if (!edits.contains(u)
      && !FSNamesystem.getSharedEditsDirs(conf).contains(u)) {
    throw new IllegalArgumentException("required edits dir " + u
        + " missing from dfs.namenode.edits.dir / shared.edits.dir");
  }
}

When it happens

Trigger: FSNamesystem construction with dfs.namenode.edits.dir.required containing a URI absent from dfs.namenode.edits.dir and dfs.namenode.shared.edits.dir — e.g. 'file:///dfs/edits' vs 'file:/dfs/edits', different hostname, or a plain typo.

Common situations: Hardening configs copied from docs where the required list names a dir never added to the main list; URI normalization mismatches (file:/// vs file:/, trailing slash); moving journals to dedicated disks.

Understand the failure class

Background: Config validation failed: what "invalid value for {key}" and settings-rejection errors mean across 19 open-source libraries — this error's family across 19 libraries.

Related errors

• Remote NameNodes not correctly configured!
• Unknown nameservice: {}
• Configuration has multiple addresses that match local node's
• Configuration dfs.namenode.rpc-address must be suffixed with
• Incompatible node types: storageType={storageType} but Stora

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1d1f28e158b45670. Report an issue: GitHub.