What causes "Failed to serialize PK index key" in apache/iceberg?

Retry the maintenance job — in-memory serialization failures are typically transient.

apache/iceberg · error · UncheckedIOException

Failed to serialize PK index key

Error message

Failed to serialize PK index key

What it means

StructLikeSerializer.serializeKey writes each primary-key field of a StructLike key into a DataOutputStream; IOExceptions from the underlying stream are rethrown as UncheckedIOException with this message. In practice this only happens on stream/IO failure since the stream is in-memory, but it signals the PK index key could not be encoded.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/StructLikeSerializer.java:68

  private final ByteArrayOutputStream baos = new ByteArrayOutputStream();
  private final DataOutputStream dos = new DataOutputStream(baos);

  public SerializedEqualityValues serializeKey(StructLike key, Types.StructType keyType) {
    baos.reset();
    try {
      List<Types.NestedField> fields = keyType.fields();
      dos.writeInt(fields.size());
      for (Types.NestedField field : fields) {
        dos.writeInt(field.fieldId());
      }

      for (int i = 0; i < fields.size(); i++) {
        writeField(key, i, fields.get(i).type());
      }

      dos.flush();
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to serialize PK index key", e);
    }

    return new SerializedEqualityValues(baos.toByteArray());
  }

  public byte[] encodePartition(StructLike partition, Types.StructType partitionType) {
    List<Types.NestedField> fields = partitionType.fields();
    if (fields.isEmpty()) {
      return EMPTY_PARTITION;
    }

    baos.reset();
    try {
      for (int i = 0; i < fields.size(); i++) {
        writeField(partition, i, fields.get(i).type());
      }

      dos.flush();

View on GitHub (pinned to 86d9c8fc54)

Solutions

Retry the maintenance job — in-memory serialization failures are typically transient.
Inspect the wrapped IOException cause for the root problem (e.g. OOM-related stream closure).
If reproducible, report an issue with the schema/types of the PK fields involved.

Defensive patterns

Strategy: try-catch

Try / catch

try {
  key = serializer.serializeKey(pkStruct);
} catch (UncheckedIOException e) {
  LOG.error("PK key serialization failed", e.getCause());
  throw e;
}

Prevention

Monitor task managers for OOM/IO issues that underlie serialization failures.
Report reproducible failures with the PK schema to the Iceberg project.

When it happens

Trigger: An IOException while writing PK struct fields to the ByteArrayOutputStream-backed DataOutputStream inside serializeKey during delete-file index construction.

Common situations: Rare; usually indicates a JVM-level IO problem or memory pressure while buffering; may surface as a wrapped failure during maintenance table changes processing.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/8b0aed6ccad3b4fd. Report an issue: GitHub.