apache/beam · critical · RuntimeException

Failed to initialize cryptography libraries needed for…

Error message

Failed to initialize cryptography libraries needed for GroupByEncryptedKey

What it means

The encrypting DoFn's setup method initializes an HMAC-SHA256 Mac and an AES/GCM Cipher from the Base64-encoded secret in the provided HmacKey. Any failure — bad Base64, missing/unsupported JCE provider, invalid key length, or absent crypto provider — is wrapped in a RuntimeException with this message.

Solutions

Verify the secret is valid standard Base64 URL (no whitespace/newlines) and decodes to a supported key length (16/24/32 bytes for AES)
Regenerate the key at exactly 32 bytes and Base64 URL-encode it before creating the HmacKey
Run on a JVM with a full JCE provider (check Cipher.getInstance("AES/GCM/NoPadding") works in the worker image)
Inspect the wrapped cause (ex.getCause()) to pinpoint whether Base64 decoding, Mac.init, or Cipher.getInstance failed

Example fix

// before
byte[] raw = "my-secret".getBytes(); // 9 bytes, not Base64, wrong length
HmacKey key = HmacKey.of(Base64.getUrlEncoder().encodeToString(raw));
// after
byte[] keyBytes = new byte[32];
new SecureRandom().nextBytes(keyBytes);
HmacKey key = HmacKey.of(Base64.getUrlEncoder().encodeToString(keyBytes));

Defensive patterns

Strategy: validation

Validate before calling

byte[] secret = Base64.getUrlDecoder().decode(secretB64);
if (secret.length != 16 && secret.length != 24 && secret.length != 32)
  throw new IllegalArgumentException("AES key must be 16/24/32 bytes");
javax.crypto.Mac.getInstance("HmacSHA256"); // verify provider availability pre-deploy

Try / catch

try {
  result = input.apply(GroupByEncryptedKey.of(hmacKey));
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("Failed to initialize cryptography")) {
    throw new IllegalStateException("Check secret Base64 encoding/key length/JCE provider", e);
  }
  throw e;
}

Prevention

Generate 32-byte keys and always Base64 URL-encode them
Test key setup in a plain JVM unit test before pipeline submission
Use the same key material on encrypt and decrypt sides

When it happens

Trigger: Beam calls setup() when the DoFn starts on a worker; it throws if getSecretBytes() is not valid Base64 URL, the decoded bytes are not a valid AES/HMAC key length (e.g. wrong size for AES-128/192/256), or Cipher.getInstance("AES/GCM/NoPadding") / Mac.getInstance("HmacSHA256") is unavailable in the JVM.

Common situations: Passing a raw (non-Base64) secret string into the HmacKey; truncated or padded-incorrectly Base64; running on a JVM with restricted JCE policy or a stripped-down runtime lacking AES/GCM; key bytes of an unsupported length (e.g. 7 bytes).

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b8e9949d4c7f856c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupByEncryptedKey.java:162

    private transient SecretKeySpec secretKeySpec;
    private transient java.security.SecureRandom generator;

    EncryptMessage(Secret hmacKey, Coder<K> keyCoder, Coder<V> valueCoder) {
      this.hmacKey = hmacKey;
      this.keyCoder = keyCoder;
      this.valueCoder = valueCoder;
    }

    @Setup
    public void setup() {
      try {
        byte[] secretBytes = java.util.Base64.getUrlDecoder().decode(this.hmacKey.getSecretBytes());
        this.mac = Mac.getInstance("HmacSHA256");
        this.mac.init(new SecretKeySpec(secretBytes, "HmacSHA256"));
        this.cipher = Cipher.getInstance("AES/GCM/NoPadding");
        this.secretKeySpec = new SecretKeySpec(secretBytes, "AES");
      } catch (Exception ex) {
        throw new RuntimeException(
            "Failed to initialize cryptography libraries needed for GroupByEncryptedKey", ex);
      }
      this.generator = new java.security.SecureRandom();
    }

    @ProcessElement
    public void processElement(ProcessContext c) throws Exception {
      byte[] encodedKey = encode(this.keyCoder, c.element().getKey());
      byte[] encodedValue = encode(this.valueCoder, c.element().getValue());

      byte[] hmac = this.mac.doFinal(encodedKey);

      byte[] keyIv = new byte[12];
      byte[] valueIv = new byte[12];
      this.generator.nextBytes(keyIv);
      this.generator.nextBytes(valueIv);
      GCMParameterSpec gcmParameterSpec = new GCMParameterSpec(128, keyIv);
      this.cipher.init(Cipher.ENCRYPT_MODE, this.secretKeySpec, gcmParameterSpec);

View on GitHub (pinned to 12126d8942)