apache/hadoop · error · IOException

Invalid configuration: a shared edits dir must not be specif

Error message

Invalid configuration: a shared edits dir must not be specified if HA is not enabled.

What it means

FSNamesystem's HA sanity check: a shared edits directory (dfs.namenode.shared.edits.dir, the journal an active NN shares with its standby) only makes sense when HA is enabled for the nameservice. If HA is off and the property is set, configuration is rejected at startup. HA being 'enabled' is derived from dfs.nameservices plus dfs.ha.namenodes.<nameservice> entries; missing those while keeping the shared dir trips this check.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:937

      LOG.info("fsOwner                = " + fsOwner);
      LOG.info("supergroup             = " + supergroup);
      LOG.info("isPermissionEnabled    = " + isPermissionEnabled);
      LOG.info("isStoragePolicyEnabled = " + isStoragePolicyEnabled);

      // block allocation has to be persisted in HA using a shared edits directory
      // so that the standby has up-to-date namespace information
      nameserviceId = DFSUtil.getNamenodeNameServiceId(conf);
      this.haEnabled = HAUtil.isHAEnabled(conf, nameserviceId);  
      
      // Sanity check the HA-related config.
      if (nameserviceId != null) {
        LOG.info("Determined nameservice ID: " + nameserviceId);
      }
      LOG.info("HA Enabled: " + haEnabled);
      if (!haEnabled && HAUtil.usesSharedEditsDir(conf)) {
        LOG.warn("Configured NNs:\n" + DFSUtil.nnAddressesAsString(conf));
        throw new IOException("Invalid configuration: a shared edits dir " +
            "must not be specified if HA is not enabled.");
      }

      // block manager needs the haEnabled initialized
      this.blockManager = new BlockManager(this, haEnabled, conf);
      this.datanodeStatistics = blockManager.getDatanodeManager().getDatanodeStatistics();

      // Get the checksum type from config
      String checksumTypeStr = conf.get(DFS_CHECKSUM_TYPE_KEY,
          DFS_CHECKSUM_TYPE_DEFAULT);
      this.isSnapshotTrashRootEnabled = conf.getBoolean(
          DFS_NAMENODE_SNAPSHOT_TRASHROOT_ENABLED,
          DFS_NAMENODE_SNAPSHOT_TRASHROOT_ENABLED_DEFAULT);
      DataChecksum.Type checksumType;
      try {
         checksumType = DataChecksum.Type.valueOf(checksumTypeStr);
      } catch (IllegalArgumentException iae) {
         throw new IOException("Invalid checksum type in "

View on GitHub (pinned to 2add963021)

Solutions

1. If staying non-HA: remove/comment dfs.namenode.shared.edits.dir from hdfs-site.xml and restart
2. If HA is intended: complete the config — dfs.nameservices, dfs.ha.namenodes.<nameservice>, and per-NN rpc addresses
3. Read the log lines just above the throw ('Determined nameservice ID', 'HA Enabled: false') to see how the NN classified the config
4. Follow the documented HA migration steps rather than hand-merging configs

Example fix

<!-- before: shared edits dir set but HA not enabled -->
<property><name>dfs.namenode.shared.edits.dir</name><value>qjournal://nn1:8485;nn2:8485/ns1</value></property>
<!-- after (non-HA): drop the property entirely -->
<!-- after (HA): also add -->
<property><name>dfs.nameservices</name><value>ns1</value></property>
<property><name>dfs.ha.namenodes.ns1</name><value>nn1,nn2</value></property>

String nsId = DFSUtil.getNamenodeNameServiceId(conf);
if (HAUtil.usesSharedEditsDir(conf)
    && !HAUtil.isHAEnabled(conf, nsId)) {
  throw new IllegalArgumentException(
      "dfs.namenode.shared.edits.dir set but HA is not enabled for " + nsId);
}

When it happens

Trigger: Starting a NameNode with dfs.namenode.shared.edits.dir configured but HAUtil.isHAEnabled(...) false: HA config removed or incomplete (missing dfs.nameservices / dfs.ha.namenodes.<ns>), or an HA nameservice converted back to single-node while the shared dir property survived.

Common situations: Leftover shared-edits config after dismantling HA; HA migration done halfway (shared journal set before the HA nameservice config); running bootstrapStandby-style tooling against a non-HA cluster.

Understand the failure class

Background: Config validation failed: what "invalid value for {key}" and settings-rejection errors mean across 19 open-source libraries — this error's family across 19 libraries.

Related errors

• Remote NameNodes not correctly configured!
• Required edits directory {} not found: dfs.namenode.edits.di
• Invalid checksum type in dfs.checksum.type: {}
• dfs.namenode.lazypersist.file.scrub.interval.sec must be zer
• Cannot find any valid remote NN to service request!

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/41d019ba41875250. Report an issue: GitHub.