{"record":{"id":"f4e5fda9f8a002a7","repo":"apache/iceberg","slug":"fail-to-deserialize-data-statistics-f4e5fd","errorCode":null,"errorMessage":"Fail to deserialize data statistics","messagePattern":"Fail to deserialize data statistics","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/StatisticsUtil.java","lineNumber":59,"sourceCode":"\n  static byte[] serializeDataStatistics(\n      DataStatistics dataStatistics, TypeSerializer<DataStatistics> statisticsSerializer) {\n    DataOutputSerializer out = new DataOutputSerializer(64);\n    try {\n      statisticsSerializer.serialize(dataStatistics, out);\n      return out.getCopyOfBuffer();\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Fail to serialize data statistics\", e);\n    }\n  }\n\n  static DataStatistics deserializeDataStatistics(\n      byte[] bytes, TypeSerializer<DataStatistics> statisticsSerializer) {\n    DataInputDeserializer input = new DataInputDeserializer(bytes, 0, bytes.length);\n    try {\n      return statisticsSerializer.deserialize(input);\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Fail to deserialize data statistics\", e);\n    }\n  }\n\n  static byte[] serializeCompletedStatistics(\n      CompletedStatistics completedStatistics,\n      TypeSerializer<CompletedStatistics> statisticsSerializer) {\n    try {\n      DataOutputSerializer out = new DataOutputSerializer(1024);\n      statisticsSerializer.serialize(completedStatistics, out);\n      return out.getCopyOfBuffer();\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Fail to serialize aggregated statistics\", e);\n    }\n  }\n\n  static CompletedStatistics deserializeCompletedStatistics(\n      byte[] bytes, CompletedStatisticsSerializer statisticsSerializer) {\n    try {","sourceCodeStart":41,"sourceCodeEnd":77,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/StatisticsUtil.java#L41-L77","documentation":"StatisticsUtil.deserializeDataStatistics wraps IOException from the TypeSerializer when deserializing a byte[] back into a DataStatistics object. It is thrown as UncheckedIOException because the input bytes could not be interpreted by the given serializer, typically indicating corrupt or incompatible data.","triggerScenarios":"Calling StatisticsUtil.deserializeDataStatistics(bytes, serializer) where the bytes were produced by a different serializer version or truncated/corrupted, so statisticsSerializer.deserialize(input) throws IOException.","commonSituations":"Restoring a Flink job checkpoint/savepoint from an older or newer Iceberg version; manually patched or truncated state bytes; network channel data mismatch after upgrade.","solutions":["Ensure the same TypeSerializer version that produced the bytes is used for deserialization (align Iceberg/Flink versions across the job).","Inspect e.getCause() for the exact deserialization offset failure to confirm byte corruption or version drift.","If restoring from an old savepoint, use the versioned migration path (CompletedStatisticsSerializer.changeSortKeySerializerVersion) or discard incompatible state.","Regenerate statistics by restarting the shuffle stage without restored state."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"if (bytes == null || bytes.length == 0) throw new IllegalArgumentException(\"statistics bytes are empty\");","typeGuard":null,"tryCatchPattern":"try {\n  DataStatistics stats = StatisticsUtil.deserializeDataStatistics(bytes, serializer);\n} catch (UncheckedIOException e) {\n  LOG.error(\"statistics deserialization failed; check version alignment\", e.getCause());\n  throw e;\n}","preventionTips":["Never mix Iceberg/Flink versions within a running job or during restore.","Validate byte payload integrity before deserialization in custom code paths.","Prefer reserialization through the supported checkpoint mechanism over hand-editing state."],"tags":["flink","deserialization","unchecked-io","shuffle"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}