{"record":{"id":"7607d56732a996b2","repo":"apache/iceberg","slug":"failed-to-deserialize-sort-key-sketch-7607d5","errorCode":null,"errorMessage":"Failed to deserialize sort key sketch","messagePattern":"Failed to deserialize sort key sketch","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySketchSerializer.java","lineNumber":113,"sourceCode":"\n    long offset = startingOffset;\n    Util.checkBounds(offset, Integer.BYTES, mem.getCapacity());\n    int numBytes = mem.getInt(offset);\n    offset += Integer.BYTES;\n\n    Util.checkBounds(offset, numBytes, mem.getCapacity());\n    byte[] sortKeyBytes = new byte[numBytes];\n    mem.getByteArray(offset, sortKeyBytes, 0, numBytes);\n    input.setBuffer(sortKeyBytes);\n\n    try {\n      List<SortKey> sortKeys = listSerializer.deserialize(input);\n      SortKey[] array = new SortKey[numItems];\n      sortKeys.toArray(array);\n      input.releaseArrays();\n      return array;\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Failed to deserialize sort key sketch\", e);\n    }\n  }\n\n  @Override\n  public int sizeOf(SortKey item) {\n    return serializeToByteArray(item).length;\n  }\n\n  @Override\n  public int sizeOf(Memory mem, long offset, int numItems) {\n    Preconditions.checkArgument(mem != null, \"Invalid input memory: null\");\n    if (numItems <= 0) {\n      return 0;\n    }\n\n    Util.checkBounds(offset, Integer.BYTES, mem.getCapacity());\n    int numBytes = mem.getInt(offset);\n    return Integer.BYTES + numBytes;","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySketchSerializer.java#L95-L131","documentation":"Thrown by SortKeySketchSerializer when deserializing a SortKey sketch from its byte representation fails with an IOException during Flink state/network transfer of shuffle statistics. It wraps the underlying serialization failure as an UncheckedIOException. The library throws it because sort-key sketches are exchanged between Flink subtasks and a corrupt or incompatible payload cannot be decoded.","triggerScenarios":"Calling deserializeFromMemory on bytes written by an incompatible SortKeySerializer version, truncated or corrupted Flink state / operator-channel payload, or reading a snapshot/checkpoint saved by a different Iceberg/Flink version whose sketch layout changed.","commonSituations":"Restoring a Flink job from a checkpoint/savepoint produced with an older Iceberg version, network-serialized statistics crossing serializer versions after an upgrade, or manual replay of statistics payloads stored out of band.","solutions":["Align the Iceberg version used to write the payload and the one deserializing it; restore checkpoints with the same version that produced them.","Check SortKeySerializer version compatibility; if restoring from a lower version, switch the serializer to v1 as StatisticsUtil does via changeSortKeySerializerVersion(1).","Re-run the shuffle from a fresh job (discard the old state) so sketches are rebuilt from input data.","Inspect the cause (IOException) for corruption/truncation and re-transfer or re-serialize the sketch."],"exampleFix":"// before: deserializing bytes written by an older serializer\nSortKey[] keys = serializer.deserializeFromMemory(record);\n// after: guard version before deserializing (as StatisticsUtil does)\nserializer.changeSortKeySerializerVersion(1);\nSortKey[] keys = serializer.deserializeFromMemory(record);\nserializer.changeSortKeySerializerVersionLatest();","handlingStrategy":"try-catch","validationCode":"if (bytes == null || bytes.length == 0) { throw new IllegalArgumentException(\"Empty sketch payload\"); }","typeGuard":"boolean isSketchPayload(byte[] bytes) { return bytes != null && bytes.length > 0; }","tryCatchPattern":"try {\n  SortKey[] keys = serializer.deserializeFromMemory(record);\n} catch (UncheckedIOException e) {\n  LOG.warn(\"Sketch deserialization failed, rebuilding statistics\", e);\n  // fall back to re-collecting statistics\n}","preventionTips":["Keep the same Iceberg version for writing and reading checkpoints/statistics payloads.","Use the v1 fallback path (changeSortKeySerializerVersion) when restoring older state.","Never hand-edit or truncate serialized statistics bytes."],"tags":["serialization","flink","unchecked-io"],"backgroundTag":"incompatible-source-type","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}