apache/hadoop · error · IllegalArgumentException

FileSystem {} is not a DFS.

Error message

FileSystem {} is not a DFS.

What it means

HAUtil.getAddressOfActive requires an HDFS handle: it immediately checks fs instanceof DistributedFileSystem and throws IllegalArgumentException otherwise, because it needs to inspect dfs.getConf()/dfs.getUri(), resolve the nameservice and probe every NN proxy for the active one.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HAUtil.java:271

    // Check whether the failover proxy provider uses logical URI.
    return provider.useLogicalURI();
  }

  /**
   * Get the internet address of the currently-active NN. This should rarely be
   * used, since callers of this method who connect directly to the NN using the
   * resulting InetSocketAddress will not be able to connect to the active NN if
   * a failover were to occur after this method has been called.
   * 
   * @param fs the file system to get the active address of.
   * @return the internet address of the currently-active NN.
   * @throws IOException if an error occurs while resolving the active NN.
   */
  public static InetSocketAddress getAddressOfActive(FileSystem fs)
      throws IOException {
    InetSocketAddress inAddr = null;
    if (!(fs instanceof DistributedFileSystem)) {
      throw new IllegalArgumentException("FileSystem " + fs + " is not a DFS.");
    }
    // force client address resolution.
    fs.exists(new Path("/"));
    DistributedFileSystem dfs = (DistributedFileSystem) fs;
    Configuration dfsConf = dfs.getConf();
    URI dfsUri = dfs.getUri();
    String nsId = dfsUri.getHost();
    if (isHAEnabled(dfsConf, nsId)) {
      List<ClientProtocol> namenodes =
          getProxiesForAllNameNodesInNameservice(dfsConf, nsId);
      for (ClientProtocol proxy : namenodes) {
        try {
          if (proxy.getHAServiceState().equals(HAServiceState.ACTIVE)) {
            inAddr = RPC.getServerAddress(proxy);
          }
        } catch (Exception e) {
          //Ignore the exception while connecting to a namenode.
          LOG.debug("Error while connecting to namenode", e);

View on GitHub (pinned to 2add963021)

Solutions

1. Construct the FileSystem from an explicit HDFS URI: FileSystem.get(URI.create("hdfs://" + nameservice), conf)
2. Set fs.defaultFS to the HA nameservice URI (e.g. hdfs://mycluster) before obtaining the FileSystem
3. Guard with instanceof DistributedFileSystem before calling getAddressOfActive

Example fix

// before
FileSystem fs = FileSystem.get(conf);              // may be LocalFileSystem
HAUtil.getAddressOfActive(fs);
// after
FileSystem fs = FileSystem.get(URI.create("hdfs://ns1"), conf);
if (!(fs instanceof DistributedFileSystem)) {
  throw new IllegalStateException("expected an HDFS FileSystem, got " + fs.getUri());
}
HAUtil.getAddressOfActive(fs);

if (!"hdfs".equalsIgnoreCase(fs.getUri().getScheme())) {
  throw new IllegalArgumentException(
      "HAUtil.getAddressOfActive needs an hdfs:// FileSystem, got " + fs.getUri());
}

static DistributedFileSystem asDistributedFileSystem(FileSystem fs) {
  return (fs instanceof DistributedFileSystem)
      ? (DistributedFileSystem) fs
      : null;
}

// usage
DistributedFileSystem dfs = asDistributedFileSystem(fs);
if (dfs == null) { /* handle non-HDFS fs */ }

catch (IllegalArgumentException e) at the call site; the message names the actual FileSystem class - report it and fix the URI/defaultFS instead of catching repeatedly.

When it happens

Trigger: Calling HAUtil.getAddressOfActive(fs) with any non-HDFS FileSystem: LocalFileSystem (file:///), RawLocalFileSystem, WebHdfsFileSystem, ViewFileSystem, S3A, etc. Happens when fs.defaultFS is not hdfs:// or when the URI scheme was left to default.

Common situations: Generic tools written against the FileSystem API calling this HA helper; unit tests that run with LocalFileSystem; fs.defaultFS misconfigured (or unset, defaulting to file:///) so FileSystem.get(conf) returns a local handle.

Understand the failure class

Background: Invalid argument type errors: "must be of type string", "expected X, got Y", and ERR_INVALID_ARG_TYPE explained — this error's family across 15 libraries.

Related errors

• {} does not support method msync
• Port {} specified in URI {} but host '{}' is a logical (HA)
• Uri without authority: {uri}
• Wrong FS: {path}, expected: {this.getUri()}
• Wrong FS: {path} and port={thatPort}, expected: {this.getUri

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ef6fbe8d0073292c. Report an issue: GitHub.