apache/hadoop · error · IOException

Unknown nameservice: {}

Error message

Unknown nameservice: {}

What it means

Thrown by DFSUtil.getParentNameServices (DFSUtil.java:726) while computing the internal nameservice list. When dfs.internal.nameservices is configured, every ID in it is validated against the full dfs.nameservices list; an ID that does not appear there is rejected with 'Unknown nameservice: <nsId>'. Both NameNode-side DFSUtil code and JournalNode (JournalNode.java:169) read dfs.internal.nameservices, so the check runs on startup of these services.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java:742

  }

  private static Collection<String> getParentNameServices(Configuration conf)
      throws IOException {
    Collection<String> parentNameServices = conf.getTrimmedStringCollection(
        DFSConfigKeys.DFS_INTERNAL_NAMESERVICES_KEY);

    if (parentNameServices.isEmpty()) {
      parentNameServices = conf.getTrimmedStringCollection(
          DFSConfigKeys.DFS_NAMESERVICES);
    } else {
      // Ensure that the internal service is indeed in the list of all available
      // nameservices.
      Collection<String> namespaces = conf
          .getTrimmedStringCollection(DFSConfigKeys.DFS_NAMESERVICES);
      Set<String> availableNameServices = new HashSet<>(namespaces);
      for (String nsId : parentNameServices) {
        if (!availableNameServices.contains(nsId)) {
          throw new IOException("Unknown nameservice: " + nsId);
        }
      }
    }

    return parentNameServices;
  }

  /**
   * Map a logical namenode ID to its lifeline address.  Use the given
   * nameservice if specified, or the configured one if none is given.
   *
   * @param conf Configuration
   * @param nsId which nameservice nnId is a part of, optional
   * @param nnId the namenode ID to get the service addr for
   * @return the lifeline addr, null if it could not be determined
   */
  public static String getNamenodeLifelineAddr(final Configuration conf,
      String nsId, String nnId) {

View on GitHub (pinned to 2add963021)

Solutions

1. Add the missing nameservice ID to dfs.nameservices so it contains every ID listed in dfs.internal.nameservices
2. Fix the typo/whitespace in the offending dfs.internal.nameservices entry so both keys agree
3. Remove decommissioned or renamed nameservice IDs from dfs.internal.nameservices
4. Restart the NameNode/JournalNode after correcting the configuration

Example fix

// before
<property><name>dfs.internal.nameservices</name><value>ns1,ns2</value></property>
<property><name>dfs.nameservices</name><value>ns1</value></property>
// after
<property><name>dfs.internal.nameservices</name><value>ns1,ns2</value></property>
<property><name>dfs.nameservices</name><value>ns1,ns2</value></property>

import org.apache.hadoop.conf.Configuration;
import java.util.*;

static void validateNameservices(Configuration conf) throws IOException {
  Set<String> all = new HashSet<>(conf.getTrimmedStringCollection("dfs.nameservices"));
  for (String ns : conf.getTrimmedStringCollection("dfs.internal.nameservices")) {
    if (!all.contains(ns)) {
      throw new IOException("Unknown nameservice: " + ns
          + " - add it to dfs.nameservices or fix dfs.internal.nameservices");
    }
  }
}

catch (IOException e) on NameNode/JournalNode startup when the message starts with 'Unknown nameservice:'; fail fast and print both dfs.internal.nameservices and dfs.nameservices values in the error report instead of retrying.

When it happens

Trigger: dfs.internal.nameservices contains an ID that is missing from (or misspelled vs) dfs.nameservices, e.g. dfs.internal.nameservices=ns1,ns2 while dfs.nameservices=ns1. Occurs when building the internal RPC address map in a federated setup.

Common situations: Federation/Router onboarding where an internal nameservice was added to dfs.internal.nameservices but dfs.nameservices was not updated; typos or whitespace in nameservice IDs; stale IDs left after decommissioning a nameservice.

Related errors

• Configuration has multiple addresses that match local node's
• Configuration dfs.namenode.rpc-address must be suffixed with
• Incompatible node types: storageType={storageType} but Stora
• All specified directories are not accessible or do not exist
• Required edits directory {} not found: dfs.namenode.edits.di

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/99ecffa0b893e910. Report an issue: GitHub.