apache/iceberg · error · UncheckedIOException

Failed to serialize PK index key

Error message

Failed to serialize PK index key

What it means

StructLikeSerializer.serializeKey wraps any IOException thrown while writing a primary-key index key (a StructLike) into an UncheckedIOException. Since serialization to a ByteArrayOutputStream should not fail in practice, this signals an unexpected internal failure, typically from the underlying DataOutput writer.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/StructLikeSerializer.java:68

  private final ByteArrayOutputStream baos = new ByteArrayOutputStream();
  private final DataOutputStream dos = new DataOutputStream(baos);

  public SerializedEqualityValues serializeKey(StructLike key, Types.StructType keyType) {
    baos.reset();
    try {
      List<Types.NestedField> fields = keyType.fields();
      dos.writeInt(fields.size());
      for (Types.NestedField field : fields) {
        dos.writeInt(field.fieldId());
      }

      for (int i = 0; i < fields.size(); i++) {
        writeField(key, i, fields.get(i).type());
      }

      dos.flush();
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to serialize PK index key", e);
    }

    return new SerializedEqualityValues(baos.toByteArray());
  }

  public byte[] encodePartition(StructLike partition, Types.StructType partitionType) {
    List<Types.NestedField> fields = partitionType.fields();
    if (fields.isEmpty()) {
      return EMPTY_PARTITION;
    }

    baos.reset();
    try {
      for (int i = 0; i < fields.size(); i++) {
        writeField(partition, i, fields.get(i).type());
      }

      dos.flush();

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the wrapped cause IOException for the actual failing field/type.
2. Confirm the table's primary key schema contains only types supported by the serializer (check writeField switch).
3. Upgrade Iceberg; if reproducible on supported types, file a bug with the schema.

// Ensure all PK field types are in the supported set before enabling upsert
schema.columns().stream().map(Types.NestedField::type).forEach(t -> check(supported.contains(t), "Unsupported PK type: " + t));

try { key = serializer.serializeKey(pk); } catch (UncheckedIOException e) { LOG.error("PK key serialization failed", e.getCause()); throw e; }

When it happens

Trigger: Calling serializeKey on a StructLike whose fields cannot be written by writeField (e.g. an unsupported/odd field type or an internal writer error) causing an IOException mid-write.

Common situations: Using Flink sink upsert/primary-key flows where the PK struct contains field types not handled by the serializer path, or internal buffer issues.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• The Avro schema is not a nullable type: ${schema}
• Fail to serialize at field: %s.
• Unrecognized version or corrupt state: <version>
• Failed to encode partition
• Unknown serialize version: ${version}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/4a76c2ee312c5c7f. Report an issue: GitHub.