apache/iceberg · error · UncheckedIOException

Fail to deserialize data statistics

Error message

Fail to deserialize data statistics

What it means

UncheckedIOException thrown by StatisticsUtil.deserializeDataStatistics when the stored bytes cannot be read back into a DataStatistics object using the supplied TypeSerializer. The wrapped IOException usually indicates truncated, corrupt, or version-mismatched state bytes.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/StatisticsUtil.java:59

  static byte[] serializeDataStatistics(
      DataStatistics dataStatistics, TypeSerializer<DataStatistics> statisticsSerializer) {
    DataOutputSerializer out = new DataOutputSerializer(64);
    try {
      statisticsSerializer.serialize(dataStatistics, out);
      return out.getCopyOfBuffer();
    } catch (IOException e) {
      throw new UncheckedIOException("Fail to serialize data statistics", e);
    }
  }

  static DataStatistics deserializeDataStatistics(
      byte[] bytes, TypeSerializer<DataStatistics> statisticsSerializer) {
    DataInputDeserializer input = new DataInputDeserializer(bytes, 0, bytes.length);
    try {
      return statisticsSerializer.deserialize(input);
    } catch (IOException e) {
      throw new UncheckedIOException("Fail to deserialize data statistics", e);
    }
  }

  static byte[] serializeCompletedStatistics(
      CompletedStatistics completedStatistics,
      TypeSerializer<CompletedStatistics> statisticsSerializer) {
    try {
      DataOutputSerializer out = new DataOutputSerializer(1024);
      statisticsSerializer.serialize(completedStatistics, out);
      return out.getCopyOfBuffer();
    } catch (IOException e) {
      throw new UncheckedIOException("Fail to serialize aggregated statistics", e);
    }
  }

  static CompletedStatistics deserializeCompletedStatistics(
      byte[] bytes, CompletedStatisticsSerializer statisticsSerializer) {
    try {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Use the same serializer that produced the bytes (same Iceberg/Flink version)
2. Confirm the bytes are non-empty and were produced by serializeDataStatistics
3. Re-create the statistics from the source (reset operator state) if the checkpoint bytes are from an incompatible version
4. Inspect the wrapped IOException cause to pinpoint the format mismatch

if (bytes == null || bytes.length == 0) { throw new IllegalArgumentException("empty statistics bytes"); }

boolean hasStatisticsBytes(byte[] bytes) { return bytes != null && bytes.length > 0; }

try { DataStatistics s = StatisticsUtil.deserializeDataStatistics(bytes, serializer); } catch (UncheckedIOException e) { /* recreate statistics or fail restore */ }

When it happens

Trigger: Calling StatisticsUtil.deserializeDataStatistics with bytes produced by a different serializer version or a different DataStatistics format; passing empty or corrupted byte arrays.

Common situations: Restoring a Flink job from a checkpoint written by an older Iceberg release; state bytes that were re-encoded or truncated; using the wrong serializer instance (e.g., completed-statistics serializer for data statistics).

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• Failed to decode partition
• Failed to deserialize sort key sketch
• Fail to deserialize aggregated statistics,change to v1
• Failed to deserialize sort key sketch
• Fail to deserialize data statistics

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/28262b5a77baa6de. Report an issue: GitHub.