apache/hadoop · error · PathIOException

EncryptionContext not present in GetPathStatus response head

Error message

EncryptionContext not present in GetPathStatus response headers

What it means

Same root cause as the other EncryptionContext errors, raised on the open-for-read path. When EncryptionType.ENCRYPTION_CONTEXT is active and the caller did not supply a VersionedFileStatus already carrying the encryption context, openFileForRead issues GetPathStatus and requires the x-ms-encryption-context response header. A null header throws PathIOException, so FS.open()/openFile() fails before any data is read.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java:997

      /*
       *  If file created with ENCRYPTION_CONTEXT, irrespective of whether isRestrictGpsOnOpenFile config is enabled or not,
       *  GetPathStatus API has to be called to get the encryptionContext from the response header
       */
      else if (getClient().getEncryptionType() == EncryptionType.ENCRYPTION_CONTEXT
              || !getAbfsConfiguration().shouldRestrictGpsOnOpenFile()) {

        AbfsHttpOperation op = getClient().getPathStatus(relativePath, false,
                tracingContext, null).getResult();
        /*
         * For file created with ENCRYPTION_CONTEXT, client shall receive
         * encryptionContext from header field: X_MS_ENCRYPTION_CONTEXT.
         */
        if (getClient().getEncryptionType() == EncryptionType.ENCRYPTION_CONTEXT) {
          final String fileEncryptionContext = op.getResponseHeader(
             HttpHeaderConfigurations.X_MS_ENCRYPTION_CONTEXT);
          if (fileEncryptionContext == null) {
            LOG.debug("EncryptionContext missing in GetPathStatus response");
            throw new PathIOException(path.toString(),
                "EncryptionContext not present in GetPathStatus response headers");
          }
          contextEncryptionAdapter = new ContextProviderEncryptionAdapter(
              getClient().getEncryptionContextProvider(), getRelativePath(path),
              fileEncryptionContext.getBytes(StandardCharsets.UTF_8));
        }
        resourceType = getClient().checkIsDir(op) ? DIRECTORY : FILE;
        contentLength = extractContentLength(op);
        eTag = op.getResponseHeader(HttpHeaderConfigurations.ETAG);
      }
      /* The only remaining case is:
       * - restrictGpsOnOpenFile config is enabled with null/wrong FileStatus and encryptionType not as ENCRYPTION_CONTEXT
       * In this case, we don't need to call GetPathStatus API.
       */
      else {
        // do nothing
      }

View on GitHub (pinned to 2add963021)

Solutions

Rewrite or copy the affected files through the ABFS filesystem with fs.azure.encryption.context.provider.type configured so the context header is persisted.
Check the file's headers directly (REST GetPathStatus) to confirm x-ms-encryption-context is absent vs. the provider being wrong.
Serve pre-encryption legacy data from a mount without the encryption-context provider.
If passing a FileStatus hint to openFile, pass a VersionedFileStatus obtained from getFileStatus under the same encrypted mount so the context travels with it.

Defensive patterns

Strategy: try-catch

Try / catch

try {
  in = fs.openFile(path).build().open();
} catch (PathIOException e) {
  if (e.getMessage().contains("GetPathStatus response headers")) {
    // missing x-ms-encryption-context: file must be rewritten under the encrypted mount
  }
}

Prevention

Verify files carry x-ms-encryption-context (raw GetPathStatus) before pointing encrypted readers at old data.
Pass a VersionedFileStatus from the same encrypted mount when using openFile hints.
Plan migrations: encrypt-then-cutover, never the reverse.

When it happens

Trigger: fs.open(path) or fs.openFile(path) with the encryption-context provider configured, where the target file lacks x-ms-encryption-context - typically files written before encryption was enabled or uploaded by non-Hadoop tools. Also triggered when callers pass a non-VersionedFileStatus or a VersionedFileStatus whose getEncryptionContext() is null (the code then falls into the GetPathStatus branch).

Common situations: Enabling encryption context on existing data lakes; jobs re-reading old output after a security retrofit; interop with files landed by AzCopy/Databricks/SDK uploads.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7a8f9b8c8d852f76. Report an issue: GitHub.