apache/hadoop · critical · IOException

No services to connect, missing NameNode address.

Error message

No services to connect, missing NameNode address.

What it means

DataNode startup/reconfiguration (BlockPoolManager.refreshNamenodes) resolved zero NameNode service-RPC addresses from the configuration — DFSUtil.getNNServiceRpcAddressesForCluster returned null/empty (any lookup IOException was logged as a warning just above). With no endpoints the DN refuses to reconfigure its block pool connections.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolManager.java:166

  void refreshNamenodes(Configuration conf)
      throws IOException {
    LOG.info("Refresh request received for nameservices: " +
        conf.get(DFSConfigKeys.DFS_NAMESERVICES));

    Map<String, Map<String, InetSocketAddress>> newAddressMap = null;
    Map<String, Map<String, InetSocketAddress>> newLifelineAddressMap = null;

    try {
      newAddressMap =
          DFSUtil.getNNServiceRpcAddressesForCluster(conf);
      newLifelineAddressMap =
          DFSUtil.getNNLifelineRpcAddressesForCluster(conf);
    } catch (IOException ioe) {
      LOG.warn("Unable to get NameNode addresses.", ioe);
    }

    if (newAddressMap == null || newAddressMap.isEmpty()) {
      throw new IOException("No services to connect, missing NameNode " +
          "address.");
    }

    synchronized (refreshNamenodesLock) {
      doRefreshNamenodes(newAddressMap, newLifelineAddressMap);
    }
  }
  
  private void doRefreshNamenodes(
      Map<String, Map<String, InetSocketAddress>> addrMap,
      Map<String, Map<String, InetSocketAddress>> lifelineAddrMap)
      throws IOException {
    assert Thread.holdsLock(refreshNamenodesLock);

    Set<String> toRefresh = new LinkedHashSet<>();
    Set<String> toAdd = new LinkedHashSet<>();
    Set<String> toRemove;
    

View on GitHub (pinned to 2add963021)

Solutions

1. Set dfs.namenode.servicerpc-address.<ns>.<nn> (or at least dfs.namenode.rpc-address) for every NameNode this DN must serve
2. For HA, define dfs.ha.namenodes.<nameservice> plus both namenode RPC addresses
3. Confirm dfs.nameservices lists the namespaces and that the DN reads the same hdfs-site.xml you edited, then restart/reload the DN

Example fix

<!-- before -->
<property><name>dfs.nameservices</name><value>ns1</value></property>
<!-- no addresses -->

<!-- after -->
<property><name>dfs.namenode.servicerpc-address.ns1.nn1</name>
  <value>nn1-host:8022</value></property>
<property><name>dfs.namenode.servicerpc-address.ns1.nn2</name>
  <value>nn2-host:8022</value></property>

Map<String, Map<String, InetSocketAddress>> addrs =
    DFSUtil.getNNServiceRpcAddressesForCluster(conf);
if (addrs == null || addrs.isEmpty()) {
  throw new IllegalStateException(
      "No NameNode service RPC addresses found; set "
      + "dfs.namenode.servicerpc-address.<ns>.<nn> and dfs.nameservices");
}

When it happens

Trigger: dfs.namenode.servicerpc-address (and its rpc-address fallback) unset for the nameservice(s); HA config missing dfs.ha.namenodes.<ns> or the per-NN addresses; dfs.nameservices/dfs.internal.nameservices empty or not matching the DN's config; unparsable hdfs-site.xml.

Common situations: DN installed with a minimal or NN-only hdfs-site.xml; federation configs where the DN's dfs.nameservices omits a namespace; typos in property suffixes (nameservice/NN names are case-sensitive).

Related errors

• Configuration dfs.namenode.rpc-address must be suffixed with
• Incompatible node types: storageType={storageType} but Stora
• Unknown nameservice: {}
• Configuration has multiple addresses that match local node's
• Unexpected configuration parameters: dfs.namenode.replicatio

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/16dc68d67a7c67db. Report an issue: GitHub.