apache/hadoop · error · IOException

Could not load failover proxy provider class {} which is con

Error message

Could not load failover proxy provider class {} which is configured for authority {}

What it means

getFailoverProxyProviderClass reads dfs.ha.failover-proxy-provider.<host> (actually configKey = prefix + '.' + host) and loads the class via Configuration.getClass; when the underlying lookup fails with ClassNotFoundException (class name on record but not loadable), this IOException reports the configured class name and the authority. Built-in providers live in hadoop-hdfs-client, so in practice the value is a typo or the class sits in a jar missing from the client classpath.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/NameNodeProxiesClient.java:294

  /** Gets the configured Failover proxy provider's class */
  @VisibleForTesting
  public static <T> Class<FailoverProxyProvider<T>> getFailoverProxyProviderClass(
      Configuration conf, URI nameNodeUri) throws IOException {
    if (nameNodeUri == null) {
      return null;
    }
    String host = nameNodeUri.getHost();
    String configKey = HdfsClientConfigKeys.Failover.PROXY_PROVIDER_KEY_PREFIX
        + "." + host;
    try {
      @SuppressWarnings("unchecked")
      Class<FailoverProxyProvider<T>> ret = (Class<FailoverProxyProvider<T>>)
          conf.getClass(configKey, null, FailoverProxyProvider.class);
      return ret;
    } catch (RuntimeException e) {
      if (e.getCause() instanceof ClassNotFoundException) {
        throw new IOException("Could not load failover proxy provider class "
            + conf.get(configKey) + " which is configured for authority "
            + nameNodeUri, e);
      } else {
        throw e;
      }
    }
  }

  /**
   * Creates an explicitly HA-enabled proxy object.
   *
   * @param conf the configuration object
   * @param nameNodeUri the URI pointing either to a specific NameNode or to a
   *        logical nameservice.
   * @param xface the IPC interface which should be created
   * @param failoverProxyProvider Failover proxy provider
   * @return an object containing both the proxy and the associated
   *         delegation token service it corresponds to

View on GitHub (pinned to 2add963021)

Solutions

Correct the class value — default should be org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
If a custom provider is intended, add its jar to the client classpath and verify the FQCN with javap/Class.forName
Remove the dfs.ha.failover-proxy-provider key entirely to use the built-in default provider

Example fix

<!-- before -->
<property>
  <name>dfs.ha.failover-proxy-providers.myNameservice</name>
  <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProviders</value>
</property> <!-- typo: Providers -->

<!-- after -->
<property>
  <name>dfs.ha.failover-proxy-providers.myNameservice</name>
  <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>

Defensive patterns

Strategy: validation

Validate before calling

String cls = conf.get("dfs.ha.failover-proxy-providers." + nameservice);
if (cls != null) {
  try {
    Class.forName(cls); // loadable on this node?
  } catch (ClassNotFoundException e) {
    throw new IllegalStateException("Provider class " + cls + " not on classpath", e);
  }
}

Try / catch

try {
  provider = NameNodeProxiesClient.getFailoverProxyProviderClass(conf, u, xface);
} catch (IOException e) {
  if (e.getMessage().contains("Could not load failover proxy provider class")) {
    // fall back to the default provider when the custom one is optional
    conf.unset("dfs.ha.failover-proxy-providers." + u.getHost());
    provider = NameNodeProxiesClient.getFailoverProxyProviderClass(conf, u, xface);
  } else throw e;
}

Prevention

Spell the provider FQCN exactly; prefer omitting the key to use the built-in default
Deploy custom provider jars to every client node and verify with Class.forName in deploy checks
After shading/relocation, update all class-name config values to match new package names

When it happens

Trigger: dfs.ha.failover-proxy-providers.<nameservice> is set to a misspelled or non-existent class; a custom provider class whose jar is absent on the node running the client; shaded/relocated deployments where the configured FQCN no longer matches relocated classes.

Common situations: Hand-edited hdfs-site.xml with wrong provider class name; custom failover logic deployed on some nodes but not the one raising the error; Maven shading that renames org.apache.hadoop classes without updating the config keys' values.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3ab5e161c6cd1990. Report an issue: GitHub.