apache/hadoop · error · HadoopIllegalArgumentException

Configuration dfs.namenode.rpc-address must be suffixed with

Error message

Configuration dfs.namenode.rpc-address must be suffixed with nameservice and namenode ID for HA configuration.

What it means

HAUtil.getNameNodeId returns dfs.ha.namenode.id if set; otherwise it tries to infer the namenode ID by matching local addresses against dfs.namenode.rpc-address.<nsId>.<nnId> via DFSUtil.getSuffixIDs with LOCAL_ADDRESS_MATCHER. If NO configured address matches this machine, the suffix lookup returns null and HadoopIllegalArgumentException is thrown demanding suffixed rpc-address configuration.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HAUtil.java:135

   * returns the namenode Id from the corresponding configuration key.
   * 
   * @param conf Configuration
   * @return namenode Id on success, null on failure.
   * @throws HadoopIllegalArgumentException on error
   */
  public static String getNameNodeId(Configuration conf, String nsId) {
    String namenodeId = conf.getTrimmed(DFS_HA_NAMENODE_ID_KEY);
    if (namenodeId != null) {
      return namenodeId;
    }
    
    String suffixes[] = DFSUtil.getSuffixIDs(conf, DFS_NAMENODE_RPC_ADDRESS_KEY,
        nsId, null, DFSUtil.LOCAL_ADDRESS_MATCHER);
    if (suffixes == null) {
      String msg = "Configuration " + DFS_NAMENODE_RPC_ADDRESS_KEY + 
          " must be suffixed with nameservice and namenode ID for HA " +
          "configuration.";
      throw new HadoopIllegalArgumentException(msg);
    }
    
    return suffixes[1];
  }

  /**
   * Similar to
   * {@link DFSUtil#getNameServiceIdFromAddress(Configuration, 
   * InetSocketAddress, String...)}
   */
  public static String getNameNodeIdFromAddress(final Configuration conf, 
      final InetSocketAddress address, String... keys) {
    // Configuration with a single namenode and no nameserviceId
    String[] ids = DFSUtil.getSuffixIDs(conf, address, keys);
    if (ids != null && ids.length > 1) {
      return ids[1];
    }
    return null;

View on GitHub (pinned to 2add963021)

Solutions

1. Set dfs.ha.namenode.id explicitly for this node, e.g. HDFS_NAMENODE_OPTS="-Ddfs.ha.namenode.id=nn1" (and dfs.nameservice.id for federated setups)
2. Fix dfs.namenode.rpc-address.<nsId>.<nnId> so this host's real address appears exactly once per nameservice
3. Verify name resolution: compare `hostname -f` output and /etc/hosts against the configured addresses

Example fix

// before: no id set and dfs.namenode.rpc-address.ns1.nn2=otherhost:8020 (not this machine)
// after
HDFS_NAMENODE_OPTS="-Ddfs.nameservice.id=ns1 -Ddfs.ha.namenode.id=nn1" hdfs --daemon start namenode

// Preflight before any HAUtil call that infers identity
String nnId = conf.getTrimmed("dfs.ha.namenode.id");
if (nnId == null) {
  // ensure one dfs.namenode.rpc-address.<nsId>.<nnId> resolves to THIS host,
  // otherwise the inference will return null and HAUtil throws
  String local = java.net.InetAddress.getLocalHost().getCanonicalHostName();
  boolean matches = false;
  for (java.util.Map.Entry<String, String> e : conf) { // iterate suffixed keys
    if (e.getKey().startsWith("dfs.namenode.rpc-address.") && e.getValue().startsWith(local)) {
      matches = true; break;
    }
  }
  if (!matches) throw new IllegalStateException(
      "Set dfs.ha.namenode.id (and dfs.nameservice.id) for host " + local);
}

catch (HadoopIllegalArgumentException e) on HA initialization; when the message names dfs.namenode.rpc-address suffixing, set dfs.ha.namenode.id explicitly rather than editing addresses by trial and error.

When it happens

Trigger: Initializing HA components without dfs.ha.namenode.id while this host's address matches none of the configured dfs.namenode.rpc-address.<nsId>.<nnId> entries (wrong host, other node's address, unresolved hostname). Note the contrast: zero matches gives this error, multiple matches gives the 'multiple addresses' error.

Common situations: First-time HA setup missing the namenode id property; config cloned from another node so rpc-addresses point elsewhere; hostname/DNS changes (hostname -f no longer equal to the configured host); starting the NameNode on the wrong machine.

Related errors

• Configuration has multiple addresses that match local node's
• Unknown nameservice: {}
• Incompatible node types: storageType={storageType} but Stora
• No services to connect, missing NameNode address.
• All specified directories are not accessible or do not exist

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e73df026372ca425. Report an issue: GitHub.