apache/iceberg · error · UncheckedIOException

Failed to serialize envelope key metadata

Error message

Failed to serialize envelope key metadata

What it means

StandardKeyMetadata.buffer() serializes this envelope key metadata (encryption key, AAD prefix, file length) into its Avro-encoded ByteBuffer form using a shared KeyMetadataEncoder. If the underlying Avro encoding raises IOException, it is rethrown as an UncheckedIOException with this message. In practice this indicates a low-level failure in the Avro binary encoding path, not a user-visible data problem.

Source

Thrown at core/src/main/java/org/apache/iceberg/encryption/StandardKeyMetadata.java:130

    }

    return parse(kmBuffer);
  }

  static StandardKeyMetadata parse(ByteBuffer buffer) {
    try {
      return KEY_METADATA_DECODER.decode(buffer);
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to parse envelope encryption metadata", e);
    }
  }

  @Override
  public ByteBuffer buffer() {
    try {
      return KEY_METADATA_ENCODER.encode(this);
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to serialize envelope key metadata", e);
    }
  }

  @Override
  public EncryptionKeyMetadata copy() {
    return new StandardKeyMetadata(this, null);
  }

  @Override
  public NativeEncryptionKeyMetadata copyWithLength(long length) {
    return new StandardKeyMetadata(this, length);
  }

  @Override
  public void put(int i, Object v) {
    switch (i) {
      case 0:
        this.encryptionKey = (ByteBuffer) v;

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Inspect the wrapped IOException cause to identify the actual Avro encoding failure.
  2. Verify the key metadata fields (encryptionKey, aadPrefix) are valid non-null ByteBuffers with correct positions.
  3. Ensure custom EncryptionKeyMetadata implementations are converted through StandardKeyMetadata.castOrParse before encoding.
  4. If reproducible, report the Avro encoder failure to the Iceberg project with the cause stack trace.

Example fix

// before — bare buffer call that may throw unchecked
ByteBuffer buf = keyMetadata.buffer();
// after — catch the unchecked wrapper and inspect cause
try {
  ByteBuffer buf = keyMetadata.buffer();
} catch (UncheckedIOException e) {
  LOG.error("key metadata encode failed", e.getCause());
  throw e;
}
Defensive patterns

Strategy: try-catch

Validate before calling

if (keyMetadata instanceof StandardKeyMetadata) return keyMetadata.buffer();
ByteBuffer b = keyMetadata.buffer();
if (b == null) throw new IllegalStateException("Null key metadata buffer");
return b;

Type guard

boolean isStandard = km instanceof StandardKeyMetadata;

Try / catch

try {
  ByteBuffer buf = keyMetadata.buffer();
} catch (UncheckedIOException e) {
  throw new IllegalStateException("key metadata serialization failed: " + e.getCause(), e);
}

Prevention

When it happens

Trigger: Calling buffer() on a StandardKeyMetadata (or via castOrParse/EncryptionKeyMetadata API) when the Avro encoder fails internally — e.g. a corrupted/reflectively-mutated internal state or an encoder I/O failure.

Common situations: Rarely hit by end users; mostly appears when custom encryption integrations supply non-standard key metadata implementations whose serialized form is incompatible, or when the shared encoder is used concurrently in ways it does not support.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/feefefe604d49269. Report an issue: GitHub.