apache/hadoop · error · IOException

Could not instantiate KeyProvider for uri: ${providerUri}

Error message

Could not instantiate KeyProvider for uri: ${providerUri}

What it means

KMSUtil.createKeyProviderFromUri delegates to KeyProviderFactory.get(uri, conf), which asks every registered provider whether it handles the URI's scheme. If none accepts it, get() returns null and KMSUtil throws IOException("Could not instantiate KeyProvider for uri: <uri>"). This is a resolution failure - the scheme matched nothing on the classpath - not a failure to reach a KMS server.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/KMSUtil.java:85

    return KMSUtil.getKeyProviderUri(
        conf, KeyProviderFactory.KEY_PROVIDER_PATH);
  }

  public static URI getKeyProviderUri(final Configuration conf,
                                      final String configKeyName) {
    final String providerUriStr = conf.getTrimmed(configKeyName);
    // No provider set in conf
    if (providerUriStr == null || providerUriStr.isEmpty()) {
      return null;
    }
    return URI.create(providerUriStr);
  }

  public static KeyProvider createKeyProviderFromUri(final Configuration conf,
      final URI providerUri) throws IOException {
    KeyProvider keyProvider = KeyProviderFactory.get(providerUri, conf);
    if (keyProvider == null) {
      throw new IOException("Could not instantiate KeyProvider for uri: " +
          providerUri);
    }
    if (keyProvider.isTransient()) {
      throw new IOException("KeyProvider " + keyProvider.toString()
          + " was found but it is a transient provider.");
    }
    return keyProvider;
  }

  @SuppressWarnings("unchecked")
  public static Map toJSON(KeyProvider.KeyVersion keyVersion) {
    Map json = new HashMap();
    if (keyVersion != null) {
      json.put(KMSRESTConstants.NAME_FIELD,
          keyVersion.getName());
      json.put(KMSRESTConstants.VERSION_NAME_FIELD,
          keyVersion.getVersionName());
      json.put(KMSRESTConstants.MATERIAL_FIELD,

View on GitHub (pinned to 2add963021)

Solutions

Verify the exact URI from config - it must be fully formed, e.g. kms://https@kms-host:9600/kms.
Confirm the KMS client jars and their META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory files are on the classpath (unzip -l the deployed jars).
For custom providers, register the service in core-site under hadoop.security.provider.services and redeploy.

Defensive patterns

Strategy: try-catch

Validate before calling

URI uri = KMSUtil.getProviderUri(conf, keyName);
if (uri == null || uri.getScheme() == null) {
  throw new IllegalArgumentException(
      "Missing or scheme-less " + keyName + " in configuration");
}

Try / catch

try {
  KeyProvider kp = KMSUtil.createKeyProviderFromUri(conf, providerUri);
} catch (IOException e) {
  // resolution failure: no registered provider accepts the URI scheme
  // fix the URI in config or the client jar on the classpath, then retry
}

Prevention

Validate the provider URI (scheme present, kms:// fully formed) at service startup.
After shading, verify META-INF/services files survived in the fat jar.

When it happens

Trigger: The provider URI in configuration (e.g. hadoop.security.key.provider.path) has an unsupported or misspelled scheme - 'kms:/https@...' missing a slash, or a bare path with no scheme - or the KMS client jar or the ServiceLoader registration for KeyProviderFactory is missing from the classpath.

Common situations: Hand-edited core-site with a malformed kms:// URI; a shaded application jar that dropped META-INF/services entries; a custom KeyProvider not registered via hadoop.security.provider.services; an older Hadoop distro missing the KMS client.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/313064cc05ea8592. Report an issue: GitHub.