What causes "Unable to load OAuth2 connection factory." in apache/hadoop?

Inspect the cause chain of the IOException to identify the exact missing key or class; the original IllegalArgumentException names the property.

apache/hadoop · error · IOException

Unable to load OAuth2 connection factory.

Error message

Unable to load OAuth2 connection factory.

What it means

URLConnectionFactory.newOAuth2URLConnectionFactory builds an SSL configurator and an OAuth2ConnectionConfigurator for WebHDFS. Any exception while creating those objects, such as a missing required OAuth2 configuration value, an invalid access-token-provider class, or an SSL configuration failure, is wrapped in this IOException. It occurs during FileSystem initialization, before a WebHDFS request is made.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java:140

    return conn;
  }

  /**
   * Construct a new URLConnectionFactory that supports OAut-based connections.
   * It will also try to load the SSL configuration when they are specified.
   */
  public static URLConnectionFactory newOAuth2URLConnectionFactory(
      int connectTimeout, int readTimeout, Configuration conf)
      throws IOException {
    ConnectionConfigurator conn;
    try {
      ConnectionConfigurator sslConnConfigurator
          = new SSLConnectionConfigurator(connectTimeout, readTimeout, conf);

      conn = new OAuth2ConnectionConfigurator(conf, sslConnConfigurator);
    } catch (Exception e) {
      throw new IOException("Unable to load OAuth2 connection factory.", e);
    }
    return new URLConnectionFactory(conn);
  }

  @VisibleForTesting
  URLConnectionFactory(ConnectionConfigurator connConfigurator) {
    this.connConfigurator = connConfigurator;
  }

  /**
   * Opens a url with read and connect timeouts
   *
   * @param url
   *          to open
   * @return URLConnection
   * @throws IOException
   */
  public URLConnection openConnection(URL url) throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

Inspect the cause chain of the IOException to identify the exact missing key or class; the original IllegalArgumentException names the property.
For the default ConfCredentialBasedAccessTokenProvider, set dfs.webhdfs.oauth2.credential, dfs.webhdfs.oauth2.client.id, and dfs.webhdfs.oauth2.refresh.url.
For ConfRefreshTokenBasedAccessTokenProvider, also set dfs.webhdfs.oauth2.refresh.token and dfs.webhdfs.oauth2.refresh.token.expires.ms.since.epoch.
Verify dfs.webhdfs.oauth2.access.token.provider names a loadable AccessTokenProvider class and validate SSL/client certificate configuration when HTTPS is used.

Example fix

// before
conf.setBoolean("dfs.webhdfs.oauth2.enabled", true);
FileSystem fs = FileSystem.get(new URI("webhdfs://nn:9870"), conf);

// after
conf.setBoolean("dfs.webhdfs.oauth2.enabled", true);
conf.set("dfs.webhdfs.oauth2.credential", credentialRef);
conf.set("dfs.webhdfs.oauth2.client.id", clientId);
conf.set("dfs.webhdfs.oauth2.refresh.url", tokenEndpointUrl);
FileSystem fs = FileSystem.get(new URI("webhdfs://nn:9870"), conf);

Defensive patterns

Strategy: validation

Validate before calling

if (conf.getBoolean("dfs.webhdfs.oauth2.enabled", false)) {
  String provider = conf.get("dfs.webhdfs.oauth2.access.token.provider",
      "org.apache.hadoop.hdfs.web.oauth2.ConfCredentialBasedAccessTokenProvider");
  requireConf(conf, "dfs.webhdfs.oauth2.client.id");
  requireConf(conf, "dfs.webhdfs.oauth2.refresh.url");
  if (provider.endsWith("ConfCredentialBasedAccessTokenProvider")) {
    requireConf(conf, "dfs.webhdfs.oauth2.credential");
  } else if (provider.endsWith("ConfRefreshTokenBasedAccessTokenProvider")) {
    requireConf(conf, "dfs.webhdfs.oauth2.refresh.token");
    requireConf(conf, "dfs.webhdfs.oauth2.refresh.token.expires.ms.since.epoch");
  }
}

Try / catch

try {
  return URLConnectionFactory.newOAuth2URLConnectionFactory(connectTimeout, readTimeout, conf);
} catch (IOException e) {
  throw new IllegalArgumentException("Invalid WebHDFS OAuth2 configuration; inspect keys starting with dfs.webhdfs.oauth2.", e);
}

Prevention

Validate all dfs.webhdfs.oauth2.* keys before constructing the FileSystem.
Never enable OAuth2 with an empty or partially copied provider configuration.
Run a small GETFILESTATUS smoke test during deployment to catch initialization-time configuration errors.

When it happens

Trigger: Set dfs.webhdfs.oauth2.enabled=true without supplying all values required by the selected dfs.webhdfs.oauth2.access.token.provider. The default credential provider requires dfs.webhdfs.oauth2.credential, dfs.webhdfs.oauth2.client.id, and dfs.webhdfs.oauth2.refresh.url; the refresh-token provider also requires dfs.webhdfs.oauth2.refresh.token and its expiry. SSL setup or provider class instantiation failures are wrapped the same way.

Common situations: OAuth2 is enabled only in the client config while the token properties were not copied into core-site.xml or the job Configuration; a custom AccessTokenProvider class is missing or throws in setConf; ssl-client.xml or truststore settings are invalid; a deployment upgrades Hadoop and configuration key names are missed.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3197aca9553576b9. Report an issue: GitHub.