apache/iceberg · error · UncheckedIOException

Failed to serialize PK index key

Error message

Failed to serialize PK index key

What it means

serializeKey() writes the equality-field ids and values of a primary-key (equality) key into Flink keyed state. Any IOException while writing the field bytes is wrapped in UncheckedIOException. In practice this means the underlying StructLike could not be converted for one of the declared key field types (e.g. an unexpected Java type for the field).

Solutions

  1. Verify the equality key field types match the actual values in the DataStream/schema; fix the schema definition passed to the maintenance source
  2. Check for schema evolution (added/renamed key fields) between the writing job and the current job and rebuild state
  3. Log the failing key's field values before this call in a debug run to find which field/value is incompatible
  4. Update to a newer Iceberg version if a type conversion gap for your field type was fixed

Example fix

// before: key field declared as TimestampType but row supplies LocalDateTime
RowType keyType = RowType.of(new TimestampType(), ...);
// after: declare the key type matching the supplied Java value
RowType keyType = RowType.of(new LocalZonedTimestampType(), ...);
Defensive patterns

Strategy: validation

Validate before calling

for (int i = 0; i < keyType.fields().size(); i++) {
  Object v = key.get(i, Object.class);
  Preconditions.checkState(v == null || Conversions.tryToByteBuffer(keyType.fields().get(i).type(), v) != null,
      "Key field %s value %s not convertible", keyType.fields().get(i).name(), v);
}

Try / catch

try {
  serializer.serializeKey(key, keyType);
} catch (UncheckedIOException e) {
  LOG.error("PK key serialization failed for key {}", key, e);
  throw e;
}

Prevention

When it happens

Trigger: Calling serializeKey with a StructLike whose field values do not match keyType's declared types so Conversions.toByteBuffer throws; a corrupted struct whose get() returns an unconvertible object.

Common situations: Row schema evolved between jobs so the equality key type no longer matches the stored rows; custom StructLike implementations returning unexpected types; upstream produced rows that violate the declared schema.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/fa8daa18aeba4058. Report an issue: GitHub.

Appendix: source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/StructLikeSerializer.java:68

  private final ByteArrayOutputStream baos = new ByteArrayOutputStream();
  private final DataOutputStream dos = new DataOutputStream(baos);

  public SerializedEqualityValues serializeKey(StructLike key, Types.StructType keyType) {
    baos.reset();
    try {
      List<Types.NestedField> fields = keyType.fields();
      dos.writeInt(fields.size());
      for (Types.NestedField field : fields) {
        dos.writeInt(field.fieldId());
      }

      for (int i = 0; i < fields.size(); i++) {
        writeField(key, i, fields.get(i).type());
      }

      dos.flush();
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to serialize PK index key", e);
    }

    return new SerializedEqualityValues(baos.toByteArray());
  }

  public byte[] encodePartition(StructLike partition, Types.StructType partitionType) {
    List<Types.NestedField> fields = partitionType.fields();
    if (fields.isEmpty()) {
      return EMPTY_PARTITION;
    }

    baos.reset();
    try {
      for (int i = 0; i < fields.size(); i++) {
        writeField(partition, i, fields.get(i).type());
      }

      dos.flush();

View on GitHub (pinned to 86d9c8fc54)