apache/hadoop · error · IOException

Multiple shared edits directories are not yet supported

Error message

Multiple shared edits directories are not yet supported

What it means

Thrown by FSNamesystem.getNamespaceEditsDirs during NameNode startup (and HA bootstrap/initializeSharedEdits) when dfs.namenode.shared.edits.dir lists more than one shared edits URI. HDFS syncs edits through a JournalSet and currently supports exactly one shared (QJM) edits directory per namespace (HDFS-2782), so requesting two is rejected outright rather than silently mis-replicating edit logs.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:1794

   */
  public static List<URI> getNamespaceEditsDirs(Configuration conf)
      throws IOException {
    return getNamespaceEditsDirs(conf, true);
  }
  
  public static List<URI> getNamespaceEditsDirs(Configuration conf,
      boolean includeShared)
      throws IOException {
    // Use a LinkedHashSet so that order is maintained while we de-dup
    // the entries.
    LinkedHashSet<URI> editsDirs = new LinkedHashSet<URI>();
    
    if (includeShared) {
      List<URI> sharedDirs = getSharedEditsDirs(conf);
  
      // Fail until multiple shared edits directories are supported (HDFS-2782)
      if (sharedDirs.size() > 1) {
        throw new IOException(
            "Multiple shared edits directories are not yet supported");
      }
  
      // First add the shared edits dirs. It's critical that the shared dirs
      // are added first, since JournalSet syncs them in the order they are listed,
      // and we need to make sure all edits are in place in the shared storage
      // before they are replicated locally. See HDFS-2874.
      for (URI dir : sharedDirs) {
        if (!editsDirs.add(dir)) {
          LOG.warn("Edits URI " + dir + " listed multiple times in " + 
              DFS_NAMENODE_SHARED_EDITS_DIR_KEY + ". Ignoring duplicates.");
        }
      }
    }    
    // Now add the non-shared dirs.
    for (URI dir : getStorageDirs(conf, DFS_NAMENODE_EDITS_DIR_KEY)) {
      if (!editsDirs.add(dir)) {
        LOG.warn("Edits URI " + dir + " listed multiple times in " + 

View on GitHub (pinned to 2add963021)

Solutions

1. Keep exactly one URI in dfs.namenode.shared.edits.dir (after de-dup, size must be <= 1) and restart the NameNode
2. Add redundancy inside that single URI by listing more JournalNodes in the quorum: qjournal://jn1:8485;jn2:8485;jn3:8485/journalId
3. Track HDFS-2782 if two independent shared storages are truly required; until then back up the QJM journal separately

Example fix

<!-- hdfs-site.xml before -->
<property><name>dfs.namenode.shared.edits.dir</name><value>qjournal://jn1:8485;jn2:8485;jn3:8485/ns1,qjournal://jn4:8485;jn5:8485/ns2</value></property>
<!-- after: one shared dir, redundancy via more JournalNodes in the quorum -->
<property><name>dfs.namenode.shared.edits.dir</name><value>qjournal://jn1:8485;jn2:8485;jn3:8485;jn4:8485/ns1</value></property>

import org.apache.hadoop.hdfs.DFSConfigKeys;
List<String> shared = conf.getTrimmedCollection(DFSConfigKeys.DFS_NAMENODE_SHARED_EDITS_DIR_KEY);
if (new HashSet<>(shared).size() > 1) {
  throw new IOException("Multiple shared edits dirs not supported: " + shared);
}

When it happens

Trigger: Setting dfs.namenode.shared.edits.dir to a comma-separated list with 2+ URIs (e.g. two qjournal:// quorums) and then starting the NameNode, or running `hdfs namenode -bootstrapStandby` / `-initializeSharedEdits`, all of which call getNamespaceEditsDirs(conf, includeShared=true).

Common situations: Admin tries to mirror edit logs to a second QJM cluster for extra redundancy; a hdfs-site.xml migration leaves the old shared dir appended after the new URI instead of replaced; copy-paste of an example config with multiple entries.

Related errors

• Configuration has multiple addresses that match local node's
• Configuration dfs.namenode.rpc-address must be suffixed with
• Interrupted waiting " + timeoutMs + "ms for a quorum of node
• Journal disabled until next roll
• Attempted to use QJM output buffer capacity (" + size + ") g

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ae300227d41bd8cc. Report an issue: GitHub.