{"record":{"id":"980842a4094bfab3","repo":"apache/iceberg","slug":"failed-to-deserialize-sort-key-sketch","errorCode":null,"errorMessage":"Failed to deserialize sort key sketch","messagePattern":"Failed to deserialize sort key sketch","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySketchSerializer.java","lineNumber":113,"sourceCode":"\n    long offset = startingOffset;\n    Util.checkBounds(offset, Integer.BYTES, mem.getCapacity());\n    int numBytes = mem.getInt(offset);\n    offset += Integer.BYTES;\n\n    Util.checkBounds(offset, numBytes, mem.getCapacity());\n    byte[] sortKeyBytes = new byte[numBytes];\n    mem.getByteArray(offset, sortKeyBytes, 0, numBytes);\n    input.setBuffer(sortKeyBytes);\n\n    try {\n      List<SortKey> sortKeys = listSerializer.deserialize(input);\n      SortKey[] array = new SortKey[numItems];\n      sortKeys.toArray(array);\n      input.releaseArrays();\n      return array;\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Failed to deserialize sort key sketch\", e);\n    }\n  }\n\n  @Override\n  public int sizeOf(SortKey item) {\n    return serializeToByteArray(item).length;\n  }\n\n  @Override\n  public int sizeOf(Memory mem, long offset, int numItems) {\n    Preconditions.checkArgument(mem != null, \"Invalid input memory: null\");\n    if (numItems <= 0) {\n      return 0;\n    }\n\n    Util.checkBounds(offset, Integer.BYTES, mem.getCapacity());\n    int numBytes = mem.getInt(offset);\n    return Integer.BYTES + numBytes;","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySketchSerializer.java#L95-L131","documentation":"UncheckedIOException thrown by SortKeySketchSerializer.deserializeFromMemory when the SortKey list inside a serialized sketch cannot be read from memory. It wraps the underlying IOException raised by Flink's list TypeSerializer, typically because the bytes are corrupt, truncated, or were written by an incompatible serializer version.","triggerScenarios":"Calling deserializeFromMemory on bytes that are truncated, corrupted, or produced by an older/newer SortKeySerializer version whose on-wire format no longer matches; also fails when numItems or the list payload does not match what the serializer expects.","commonSituations":"Restoring a Flink job from a checkpoint/savepoint taken with a different Iceberg version; corrupted or hand-edited state bytes; passing a byte array of the wrong payload to the memory deserializer.","solutions":["Verify the checkpoint/savepoint was created with the same (or compatible) Iceberg/Flink version as the restoring job","Regenerate the sketch bytes via SortKeySketchSerializer.serializeToByteArray instead of reusing old bytes","If restoring across versions, use StatisticsUtil.deserializeCompletedStatistics which falls back to sort key serializer v1","Inspect the wrapped IOException cause to identify exact offset/format mismatch"],"exampleFix":"// before\nSortKey[] keys = serializer.deserializeFromMemory(legacyBytes);\n// after\ntry {\n  SortKey[] keys = serializer.deserializeFromMemory(legacyBytes);\n} catch (UncheckedIOException e) {\n  // fall back to v1 sort key serialization for old checkpoints\n  sortKeySerializer.changeSortKeySerializerVersion(1);\n  SortKey[] keys = serializer.deserializeFromMemory(legacyBytes);\n  sortKeySerializer.changeSortKeySerializerVersionLatest();\n}","handlingStrategy":"try-catch","validationCode":"if (bytes == null || bytes.length == 0) { throw new IllegalArgumentException(\"empty sketch bytes\"); }","typeGuard":"boolean isSortableSketchBytes(byte[] bytes) { return bytes != null && bytes.length > 0; }","tryCatchPattern":"try { SortKey[] keys = serializer.deserializeFromMemory(bytes); } catch (UncheckedIOException e) { /* fall back to v1 serializer or regenerate sketch */ }","preventionTips":["Pin the same Iceberg/Flink version for job and checkpoint producer","Never hand-edit or truncate serialized sketch bytes","Regenerate sketches via serializeToByteArray after any version upgrade"],"tags":["flink","serialization","deserialization"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}