apache/iceberg · error · UncheckedIOException

Fail to deserialize data statistics

Error message

Fail to deserialize data statistics

What it means

StatisticsUtil.deserializeDataStatistics reads a DataStatistics back from bytes through the given TypeSerializer; IOException becomes UncheckedIOException "Fail to deserialize data statistics". It means the byte payload is not in the format the serializer expects (wrong version, truncation, or wrong serializer).

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/StatisticsUtil.java:59

  static byte[] serializeDataStatistics(
      DataStatistics dataStatistics, TypeSerializer<DataStatistics> statisticsSerializer) {
    DataOutputSerializer out = new DataOutputSerializer(64);
    try {
      statisticsSerializer.serialize(dataStatistics, out);
      return out.getCopyOfBuffer();
    } catch (IOException e) {
      throw new UncheckedIOException("Fail to serialize data statistics", e);
    }
  }

  static DataStatistics deserializeDataStatistics(
      byte[] bytes, TypeSerializer<DataStatistics> statisticsSerializer) {
    DataInputDeserializer input = new DataInputDeserializer(bytes, 0, bytes.length);
    try {
      return statisticsSerializer.deserialize(input);
    } catch (IOException e) {
      throw new UncheckedIOException("Fail to deserialize data statistics", e);
    }
  }

  static byte[] serializeCompletedStatistics(
      CompletedStatistics completedStatistics,
      TypeSerializer<CompletedStatistics> statisticsSerializer) {
    try {
      DataOutputSerializer out = new DataOutputSerializer(1024);
      statisticsSerializer.serialize(completedStatistics, out);
      return out.getCopyOfBuffer();
    } catch (IOException e) {
      throw new UncheckedIOException("Fail to serialize aggregated statistics", e);
    }
  }

  static CompletedStatistics deserializeCompletedStatistics(
      byte[] bytes, CompletedStatisticsSerializer statisticsSerializer) {
    try {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Use StatisticsUtil.deserializeCompletedStatistics instead, which has the v1 fallback path for sort-key statistics.
2. Restore from a state/checkpoint written by the same Iceberg version, or upgrade both writer and reader to the same runtime version.
3. Verify the serializer argument matches the statistics type that was serialized.

Example fix

// before
DataStatistics s = StatisticsUtil.deserializeDataStatistics(bytes, sortKeyStatsSerializer); // v2 reading v1 bytes
// after
CompletedStatistics s = StatisticsUtil.deserializeCompletedStatistics(bytes, completedSerializer); // handles v1 fallback

if (bytes == null || bytes.length == 0) { return null; }

try {
  return StatisticsUtil.deserializeDataStatistics(bytes, serializer);
} catch (UncheckedIOException e) {
  // fall back: rebuild statistics or use version-aware util
  return null;
}

When it happens

Trigger: Calling deserializeDataStatistics with bytes written by a different serializer version (e.g. SortKey statistics serialized with v1 SortKeySerializer but read with v2) or corrupted/truncated state bytes.

Common situations: Restoring a job across Iceberg versions where the sort-key serializer version changed; mismatched serializer argument passed to the utility.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• Failed to deserialize sort key sketch
• Fail to deserialize aggregated statistics,change to v1
• Fail to deserialize aggregated statistics,change to v1
• Fail to deserialize aggregated statistics
• Failed to decode partition

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/fa3d19da70366b0f. Report an issue: GitHub.