apache/hadoop · error · HadoopIllegalArgumentException

Configuration has multiple addresses that match local node's

Error message

Configuration has multiple addresses that match local node's address. Please configure the system with dfs.nameservice.id and dfs.ha.namenode.id

What it means

When dfs.nameservice.id and dfs.ha.namenode.id are not set, Hadoop auto-detects this node's HA identity: DFSUtil.getSuffixIDs(..., LOCAL_ADDRESS_MATCHER) matches local network addresses against the configured dfs.namenode.rpc-address.<nsId>.<nnId> entries. If more than one configured address matches the local machine, the identity is ambiguous and HadoopIllegalArgumentException is thrown instead of guessing.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java:1309

        try {
          s = NetUtils.createSocketAddr(addr);
        } catch (Exception e) {
          LOG.warn("Exception in creating socket address " + addr, e);
          continue;
        }
        if (!s.isUnresolved() && matcher.match(s)) {
          nameserviceId = nsId;
          namenodeId = nnId;
          found++;
        }
      }
    }
    if (found > 1) { // Only one address must match the local address
      String msg = "Configuration has multiple addresses that match "
          + "local node's address. Please configure the system with "
          + DFS_NAMESERVICE_ID + " and "
          + DFS_HA_NAMENODE_ID_KEY;
      throw new HadoopIllegalArgumentException(msg);
    }
    return new String[] { nameserviceId, namenodeId };
  }
  
  /**
   * For given set of {@code keys} adds nameservice Id and or namenode Id
   * and returns {nameserviceId, namenodeId} when address match is found.
   * @see #getSuffixIDs(Configuration, String, String, String, AddressMatcher)
   */
  static String[] getSuffixIDs(final Configuration conf,
      final InetSocketAddress address, final String... keys) {
    AddressMatcher matcher = new AddressMatcher() {
     @Override
      public boolean match(InetSocketAddress s) {
        return address.equals(s);
      } 
    };
    

View on GitHub (pinned to 2add963021)

Solutions

1. Set the identity explicitly for this node, e.g. HDFS_NAMENODE_OPTS="-Ddfs.nameservice.id=ns1 -Ddfs.ha.namenode.id=nn1", or use per-host config files
2. Give every dfs.namenode.rpc-address.<nsId>.<nnId> a unique host:port so only one entry can match
3. Replace 0.0.0.0/wildcard rpc-addresses with concrete addresses in HA configurations

Example fix

// before: no explicit ids and two entries match this host
//   dfs.namenode.rpc-address.ns1.nn1=host1:8020
//   dfs.namenode.rpc-address.ns2.nn1=host1:8020
// after: pin the identity when starting the NameNode
HDFS_NAMENODE_OPTS="-Ddfs.nameservice.id=ns1 -Ddfs.ha.namenode.id=nn1" hdfs --daemon start namenode

// Before starting a NameNode in HA, pin the identity so address matching is never consulted
System.setProperty("dfs.nameservice.id", "ns1");
System.setProperty("dfs.ha.namenode.id", "nn1");
// or pass -Ddfs.nameservice.id=ns1 -Ddfs.ha.namenode.id=nn1 via HDFS_NAMENODE_OPTS

catch (HadoopIllegalArgumentException e) during NN/HA initialization; when the message mentions 'multiple addresses', report that dfs.nameservice.id/dfs.ha.namenode.id must be set for this host rather than retrying.

When it happens

Trigger: Two nameservices (or two namenode IDs) configured with the same host:port or a 0.0.0.0 wildcard address that matches the local machine; starting a NameNode/HA utility on a host whose address matches multiple dfs.namenode.rpc-address entries with no explicit dfs.nameservice.id/dfs.ha.namenode.id.

Common situations: Co-located HA NameNodes or shared ports; wildcard rpc-addresses in HA configs; startup scripts that never set the HA identity JVM properties; hosts with several interfaces each matching a different configured address.

Related errors

• Configuration dfs.namenode.rpc-address must be suffixed with
• Unknown nameservice: {}
• Incompatible node types: storageType={storageType} but Stora
• All specified directories are not accessible or do not exist
• Required edits directory {} not found: dfs.namenode.edits.di

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4386c1624b6e7483. Report an issue: GitHub.