apache/hadoop · critical · PathIOException

Both global key and encryption context are set, only one all

Error message

Both global key and encryption context are set, only one allowed

What it means

On secure (https) connections ABFS supports two mutually exclusive client-side encryption mechanisms: the global client-provided encryption key (fs.azure.encryption.encoded.client-provided-key plus its -sha property) and the per-file encryption-context provider (fs.azure.encryption.context.provider.type). During client initialization, if both a provider and a global key are configured, PathIOException 'Both global key and encryption context are set, only one allowed' is thrown and the filesystem fails to initialize.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java:1883

      tokenProvider = (AccessTokenProvider) providers[0];
      sasTokenProvider = (SASTokenProvider) providers[1];
      ExtensionHelper.bind(tokenProvider, uri,
          abfsConfiguration.getRawConfiguration());
    } else {
      LOG.trace("Fetching token provider");
      tokenProvider = abfsConfiguration.getTokenProvider();
      ExtensionHelper.bind(tokenProvider, uri,
          abfsConfiguration.getRawConfiguration());
    }

    // Encryption setup
    EncryptionContextProvider encryptionContextProvider = null;
    if (isSecure) {
      encryptionContextProvider =
          abfsConfiguration.createEncryptionContextProvider();
      if (encryptionContextProvider != null) {
        if (abfsConfiguration.getEncodedClientProvidedEncryptionKey() != null) {
          throw new PathIOException(uri.getPath(),
              "Both global key and encryption context are set, only one allowed");
        }
        encryptionContextProvider.initialize(
            abfsConfiguration.getRawConfiguration(), accountName,
            fileSystemName);
      } else if (abfsConfiguration.getEncodedClientProvidedEncryptionKey() != null) {
        if (abfsConfiguration.getEncodedClientProvidedEncryptionKeySHA() == null) {
          throw new PathIOException(uri.getPath(),
              "Encoded SHA256 hash must be provided for global encryption");
        }
      }
    }

    LOG.trace("Initializing AbfsClientHandler for {}", baseUrl);
    this.clientHandler = new AbfsClientHandler(baseUrl, creds,
        abfsConfiguration,
        tokenProvider, sasTokenProvider, encryptionContextProvider,
        populateAbfsClientContext());

View on GitHub (pinned to 2add963021)

Solutions

1. Choose one mechanism: to use the encryption-context provider, delete fs.azure.encryption.encoded.client-provided-key and fs.azure.encryption.encoded.client-provided-key-sha.
2. Or remove fs.azure.encryption.context.provider.type to keep the global client-provided key.
3. Search the full effective config (core-site.xml, account-specific fs.azure.account.* keys, cluster-wide safety valves) for both property families.
4. Restart services after the change; the check runs only at client construction.

Example fix

<!-- before: both set -> PathIOException -->
<property><name>fs.azure.encryption.context.provider.type</name><value>my.EncryptionContextProvider</value></property>
<property><name>fs.azure.encryption.encoded.client-provided-key</name><value>base64key</value></property>

<!-- after: provider only -->
<property><name>fs.azure.encryption.context.provider.type</name><value>my.EncryptionContextProvider</value></property>

boolean hasProvider = conf.get("fs.azure.encryption.context.provider.type") != null;
boolean hasCppek = conf.get("fs.azure.encryption.encoded.client-provided-key") != null
    || conf.get("fs.azure.encryption.encoded.client-provided-key-sha") != null;
if (hasProvider && hasCppek) {
  throw new IllegalArgumentException("Set either the encryption-context provider or the client-provided key, not both.");
}

try { fs = FileSystem.get(conf); } catch (PathIOException e) { if (e.getMessage().contains("only one allowed")) { /* remove one encryption config family */ } throw e; }

When it happens

Trigger: A secure abfss:// mount with both fs.azure.encryption.context.provider.type and fs.azure.encryption.encoded.client-provided-key (or -sha) set - including via account-specific overlays in the same config.

Common situations: Rolling out the newer encryption-context feature on clusters that already had client-provided keys; merging encryption sections from different documentation pages; leftover CPPK keys after migrating to the provider model.

Related errors

• Encoded SHA256 hash must be provided for global encryption
• EncryptionContext not present in GetPathStatus response
• EncryptionContext not present in GetPathStatus response head
• File doesn't have encryptionContext.
• "%s" must be set for user-bound SAS auth type.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/42ed95859802e237. Report an issue: GitHub.