apache/hadoop · error · DelegationTokenIOException

Incompatible EncryptionSecrets version: {versionId}

Error message

Incompatible EncryptionSecrets version: {versionId}

What it means

EncryptionSecrets embeds the SSE key material inside S3A delegation tokens and prefixes its serialized form with a version stamp. readFields accepts only SERIAL_VERSION_UID_1 (legacy, no encryption context) and the current serialVersionUID; any other leading long is treated as an incompatible format and throws DelegationTokenIOException 'Incompatible EncryptionSecrets version'. This is a wire-format guard, not a data error.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/EncryptionSecrets.java:188

   * After reading, call {@link #init()}.
   * @param in {@code DataInput} to deserialize this object from.
   * @throws IOException failure to read/validate data.
   */
  @Override
  public void readFields(final DataInput in) throws IOException {
    final LongWritable version = new LongWritable();
    version.readFields(in);
    boolean readContext;

    final long versionId = version.get();
    if (versionId == SERIAL_VERSION_UID_1) {
      LOG.info("Unmarshalling Encryption Secrets from older client; "
          + "setting encryption context to \"\"");
      readContext = false;
    } else if (versionId == serialVersionUID) {
      readContext = true;
    } else {
      throw new DelegationTokenIOException(
          "Incompatible EncryptionSecrets version: " + versionId);
    }
    encryptionAlgorithm = Text.readString(in, MAX_SECRET_LENGTH);
    encryptionKey = Text.readString(in, MAX_SECRET_LENGTH);
    if (readContext) {
      encryptionContext = Text.readString(in);
    } else {
      encryptionContext = DEFAULT_S3_ENCRYPTION_CONTEXT;
    }
    init();
  }

  /**
   * For java serialization: read and then call {@link #init()}.
   * @param in input
   * @throws IOException IO problem
   * @throws ClassNotFoundException problem loading inner class.
   */

View on GitHub (pinned to 2add963021)

Solutions

1. Re-issue delegation tokens after upgrading so all tokens use the current format
2. Run issuer and consumers on the same hadoop-aws version (at least for delegation-token compatibility)
3. During rolling upgrades, expire old tokens before switching traffic to the new version

// no pre-check possible from token bytes alone; version is opaque until decode.
// Mitigate by comparing hadoop-aws versions before accepting a token:
String issuerVersion = tokenConf.get("hadoop.build.version"); // recorded at issue time if available
if (!VersionInfo.getVersion().equals(issuerVersion)) {
  LOG.warn("Token from hadoop {} decoded by {}; re-fetch if decode fails", issuerVersion, VersionInfo.getVersion());
}

try {
  tokens.bindToAnyDelegationToken();
} catch (DelegationTokenIOException e) {
  if (e.getMessage().contains("Incompatible EncryptionSecrets version")) {
    // version skew: discard token and re-authenticate directly
    credentials.removeAllTokens();
    deployUnbondedAndProceed();
  } else {
    throw e;
  }
}

When it happens

Trigger: A delegation token issued by a hadoop-aws build whose EncryptionSecrets serialVersionUID differs from the decoding build is decoded during token binding. The version long read from the token bytes matches neither known version.

Common situations: Rolling upgrades where executors run a different Hadoop patch line than the service that issued tokens; long-lived tokens surviving a cluster upgrade; mixed-version distros in one workflow.

Related errors

• No URI in {this}
• Delegation token is wrong class; expected a token identifier
• No AWS credential providers created by Delegation Token Bind
• Decoding S3A token {cause}
• Failed to unmarshall token for {canonicalUri}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5acbd96939fca4c3. Report an issue: GitHub.