apache/iceberg · error · UncheckedIOException

Failed to serialize PK index key

Error message

Failed to serialize PK index key

What it means

StructLikeSerializer.serializeKey writes each primary-key field of a StructLike row into a byte buffer for the equality-delete PK index. Any IOException from the underlying DataOutputStream (or from writeField) is wrapped in an UncheckedIOException 'Failed to serialize PK index key'.

Solutions

1. Inspect the wrapped cause (e.getCause()) to find which field type writeField rejected; ensure the equality-delete/PK columns use supported primitive types.
2. Re-run the maintenance index rebuild after schema changes so keys are serialized against the current schema.
3. If a new Iceberg type is involved, upgrade to an Iceberg version whose StructLikeSerializer handles it, or exclude that field from the PK index.

for (Types.NestedField f : schema.primaryKeyFields()) {
  Preconditions.checkArgument(
      f.type().isPrimitiveType(),
      "Unsupported PK field type for index key: %s", f.type());
}

try {
  SerializedEqualityValues key = serializer.serializeKey(row, pkFields);
} catch (UncheckedIOException e) {
  LOG.error("PK key serialization failed: {}", e.getCause().getMessage());
  throw e; // index consistency requires failing the cycle
}

When it happens

Trigger: serializeKey is called on a StructLike whose PK field values cannot be written by writeField — e.g. a field type not handled by writeField, or the buffer/data output failing mid-write.

Common situations: Table schema changed the PK columns' types after the index was built (e.g. adding a nested/complex type the serializer does not support); a null or unexpected value in a key field reaching an unhandled branch.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• Failed to encode partition
• Failed to serialize PK index key
• Could not deserialize the WriteResult object
• Could not deserialize the WriteResult object
• Could not deserialize the WriteResult object

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/16c9295a283e5091. Report an issue: GitHub.

Appendix: source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/StructLikeSerializer.java:68

  private final ByteArrayOutputStream baos = new ByteArrayOutputStream();
  private final DataOutputStream dos = new DataOutputStream(baos);

  public SerializedEqualityValues serializeKey(StructLike key, Types.StructType keyType) {
    baos.reset();
    try {
      List<Types.NestedField> fields = keyType.fields();
      dos.writeInt(fields.size());
      for (Types.NestedField field : fields) {
        dos.writeInt(field.fieldId());
      }

      for (int i = 0; i < fields.size(); i++) {
        writeField(key, i, fields.get(i).type());
      }

      dos.flush();
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to serialize PK index key", e);
    }

    return new SerializedEqualityValues(baos.toByteArray());
  }

  public byte[] encodePartition(StructLike partition, Types.StructType partitionType) {
    List<Types.NestedField> fields = partitionType.fields();
    if (fields.isEmpty()) {
      return EMPTY_PARTITION;
    }

    baos.reset();
    try {
      for (int i = 0; i < fields.size(); i++) {
        writeField(partition, i, fields.get(i).type());
      }

      dos.flush();

View on GitHub (pinned to 86d9c8fc54)