What causes "File doesn't have encryptionContext." in apache/hadoop?

Create with overwrite=true instead of append - the fresh file is written with a new encryption context under the configured provider.

apache/hadoop · error · PathIOException

File doesn't have encryptionContext.

Error message

File doesn't have encryptionContext.

What it means

Write-path sibling of the GetPathStatus encryption-context checks. In openFileForWrite, when the client's EncryptionType is ENCRYPTION_CONTEXT, the existing file must return x-ms-encryption-context on its status call so a ContextProviderEncryptionAdapter can be built for append/in-place write. A null header throws PathIOException 'File doesn't have encryptionContext.'

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java:1132

      final long contentLength = extractContentLength(op.getResult());
      final long offset = overwrite ? 0 : contentLength;

      perfInfo.registerSuccess(true);

      boolean isAppendBlob = false;
      if (isAppendBlobKey(path.toString())) {
        isAppendBlob = true;
      }

      AbfsLease lease = maybeCreateLease(relativePath, tracingContext);
      final String eTag = extractEtagHeader(op.getResult());
      final ContextEncryptionAdapter contextEncryptionAdapter;
      if (writeClient.getEncryptionType() == EncryptionType.ENCRYPTION_CONTEXT) {
        final String encryptionContext = op.getResult()
            .getResponseHeader(
                HttpHeaderConfigurations.X_MS_ENCRYPTION_CONTEXT);
        if (encryptionContext == null) {
          throw new PathIOException(path.toString(),
              "File doesn't have encryptionContext.");
        }
        contextEncryptionAdapter = new ContextProviderEncryptionAdapter(
            writeClient.getEncryptionContextProvider(), getRelativePath(path),
            encryptionContext.getBytes(StandardCharsets.UTF_8));
      } else {
        contextEncryptionAdapter = NoContextEncryptionAdapter.getInstance();
      }

      return new AbfsOutputStream(
          populateAbfsOutputStreamContext(
              isAppendBlob,
              lease,
              getClientHandler(),
              statistics,
              relativePath,
              offset,
              eTag,

View on GitHub (pinned to 2add963021)

Solutions

Create with overwrite=true instead of append - the fresh file is written with a new encryption context under the configured provider.
If append semantics are required, first copy the file through the encrypted ABFS mount (read+rewrite) so it gains the context, then append.
Verify with a raw GetPathStatus whether x-ms-encryption-context exists before assuming provider misconfiguration.
Keep unencrypted legacy files on a mount without fs.azure.encryption.context.provider.type.

Example fix

// before - append fails on pre-encryption file
try (FSDataOutputStream out = fs.append(path)) { ... }

// after - rewrite the file via the encrypted mount, then append
try (FSDataInputStream in = fs.open(path);
     FSDataOutputStream out = fs.create(path, true)) {
  IOUtils.copyBytes(in, out, 4 << 20, false);
}
try (FSDataOutputStream out = fs.append(path)) { ... }

Defensive patterns

Strategy: try-catch

Try / catch

try {
  out = fs.append(path);
} catch (PathIOException e) {
  if (e.getMessage().contains("encryptionContext")) {
    // file lacks x-ms-encryption-context: create(path, true) to rewrite with a fresh context
  }
}

Prevention

Do not append to files that predate encryption-context enablement.
When both append and overwrite are acceptable on encrypted mounts, prefer create(overwrite=true).
Validate append targets came from the same encrypted pipeline.

When it happens

Trigger: fs.append(path) or create(path, overwrite=false) in ENCRYPTION_CONTEXT mode on an existing file that lacks the encryption-context header (written pre-encryption or by external tools).

Common situations: Appending to logs/outputs written before the encryption-context provider was enabled; appending to files landed by AzCopy/SDK; provider rollout to existing data lakes.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3227f15b428cebcc. Report an issue: GitHub.