{"record":{"id":"fa3d19da70366b0f","repo":"apache/iceberg","slug":"fail-to-deserialize-data-statistics-fa3d19","errorCode":null,"errorMessage":"Fail to deserialize data statistics","messagePattern":"Fail to deserialize data statistics","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/StatisticsUtil.java","lineNumber":59,"sourceCode":"\n  static byte[] serializeDataStatistics(\n      DataStatistics dataStatistics, TypeSerializer<DataStatistics> statisticsSerializer) {\n    DataOutputSerializer out = new DataOutputSerializer(64);\n    try {\n      statisticsSerializer.serialize(dataStatistics, out);\n      return out.getCopyOfBuffer();\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Fail to serialize data statistics\", e);\n    }\n  }\n\n  static DataStatistics deserializeDataStatistics(\n      byte[] bytes, TypeSerializer<DataStatistics> statisticsSerializer) {\n    DataInputDeserializer input = new DataInputDeserializer(bytes, 0, bytes.length);\n    try {\n      return statisticsSerializer.deserialize(input);\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Fail to deserialize data statistics\", e);\n    }\n  }\n\n  static byte[] serializeCompletedStatistics(\n      CompletedStatistics completedStatistics,\n      TypeSerializer<CompletedStatistics> statisticsSerializer) {\n    try {\n      DataOutputSerializer out = new DataOutputSerializer(1024);\n      statisticsSerializer.serialize(completedStatistics, out);\n      return out.getCopyOfBuffer();\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Fail to serialize aggregated statistics\", e);\n    }\n  }\n\n  static CompletedStatistics deserializeCompletedStatistics(\n      byte[] bytes, CompletedStatisticsSerializer statisticsSerializer) {\n    try {","sourceCodeStart":41,"sourceCodeEnd":77,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/StatisticsUtil.java#L41-L77","documentation":"StatisticsUtil.deserializeDataStatistics reads a DataStatistics back from bytes through the given TypeSerializer; IOException becomes UncheckedIOException \"Fail to deserialize data statistics\". It means the byte payload is not in the format the serializer expects (wrong version, truncation, or wrong serializer).","triggerScenarios":"Calling deserializeDataStatistics with bytes written by a different serializer version (e.g. SortKey statistics serialized with v1 SortKeySerializer but read with v2) or corrupted/truncated state bytes.","commonSituations":"Restoring a job across Iceberg versions where the sort-key serializer version changed; mismatched serializer argument passed to the utility.","solutions":["Use StatisticsUtil.deserializeCompletedStatistics instead, which has the v1 fallback path for sort-key statistics.","Restore from a state/checkpoint written by the same Iceberg version, or upgrade both writer and reader to the same runtime version.","Verify the serializer argument matches the statistics type that was serialized."],"exampleFix":"// before\nDataStatistics s = StatisticsUtil.deserializeDataStatistics(bytes, sortKeyStatsSerializer); // v2 reading v1 bytes\n// after\nCompletedStatistics s = StatisticsUtil.deserializeCompletedStatistics(bytes, completedSerializer); // handles v1 fallback","handlingStrategy":"fallback","validationCode":"if (bytes == null || bytes.length == 0) { return null; }","typeGuard":null,"tryCatchPattern":"try {\n  return StatisticsUtil.deserializeDataStatistics(bytes, serializer);\n} catch (UncheckedIOException e) {\n  // fall back: rebuild statistics or use version-aware util\n  return null;\n}","preventionTips":["Use deserializeCompletedStatistics for sort-key statistics (has v1 fallback)","Keep Iceberg runtime versions consistent across job restarts","Confirm the serializer matches the serialized statistics type"],"tags":["flink","deserialization","unchecked-io","version-compatibility"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}