apache/iceberg · error · UncheckedIOException

Failed to encode partition

Error message

Failed to encode partition

What it means

StructLikeSerializer.encodePartition serializes each field of a partition StructLike (using the partition spec's struct type) into a byte[] used to group/compare partitions. An IOException while writing is wrapped in an UncheckedIOException 'Failed to encode partition'.

Solutions

1. Check the wrapped cause to identify the failing field type; ensure partition fields are supported primitive types.
2. Pass the correct Types.StructType for the file's specId (table.specs().get(dataFile.specId().partitionType()) style lookup) rather than the default spec's type.
3. Re-run the maintenance cycle after any partition-spec evolution so partitions are encoded with the current spec metadata.

Example fix

// before
byte[] key = serializer.encodePartition(dataFile.partition(), table.spec().partitionType());
// assumes the file uses the current spec

// after
Types.StructType pt = table.specs().get(dataFile.specId()).partitionType();
byte[] key = serializer.encodePartition(dataFile.partition(), pt);

Types.StructType partitionType = table.specs().get(dataFile.specId()).partitionType();
for (int i = 0; i < partitionType.fields().size(); i++) {
  Preconditions.checkArgument(
      partitionType.fields().get(i).type().isPrimitiveType(),
      "Unsupported partition field type: %s", partitionType.fields().get(i).type());
}

try {
  byte[] encoded = serializer.encodePartition(partition, partitionType);
} catch (UncheckedIOException e) {
  LOG.error("Partition encoding failed: {}", e.getCause().getMessage());
  throw e;
}

When it happens

Trigger: encodePartition is called with a partition StructLike and Types.StructType whose fields writeField cannot encode — typically a partition field type unsupported by writeField or an IO failure on the internal DataOutputStream.

Common situations: Partition spec evolved to include a field type the current serializer does not handle; a partition value is null where the encoder expects a value; passing the wrong partitionType (mismatched specId) so fields do not line up.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• Failed to decode partition
• Failed to encode partition
• Failed to serialize PK index key
• Failed to serialize PK index key
• Could not deserialize the WriteResult object

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/17eddde8daa9dfd3. Report an issue: GitHub.

Appendix: source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/StructLikeSerializer.java:88

    return new SerializedEqualityValues(baos.toByteArray());
  }

  public byte[] encodePartition(StructLike partition, Types.StructType partitionType) {
    List<Types.NestedField> fields = partitionType.fields();
    if (fields.isEmpty()) {
      return EMPTY_PARTITION;
    }

    baos.reset();
    try {
      for (int i = 0; i < fields.size(); i++) {
        writeField(partition, i, fields.get(i).type());
      }

      dos.flush();
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to encode partition", e);
    }

    return baos.toByteArray();
  }

  public static StructLike decodePartition(byte[] encoded, Types.StructType partitionType) {
    PartitionData partition = new PartitionData(partitionType);
    List<Types.NestedField> fields = partitionType.fields();
    if (fields.isEmpty()) {
      return partition;
    }

    try (DataInputStream dis = new DataInputStream(new ByteArrayInputStream(encoded))) {
      for (int i = 0; i < fields.size(); i++) {
        boolean isNull = dis.readBoolean();
        if (isNull) {
          partition.set(i, null);
        } else {

View on GitHub (pinned to 86d9c8fc54)