{"record":{"id":"080555189226ca83","repo":"apache/iceberg","slug":"failed-to-deserialize-sort-key-sketch-080555","errorCode":null,"errorMessage":"Failed to deserialize sort key sketch","messagePattern":"Failed to deserialize sort key sketch","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySketchSerializer.java","lineNumber":113,"sourceCode":"\n    long offset = startingOffset;\n    Util.checkBounds(offset, Integer.BYTES, mem.getCapacity());\n    int numBytes = mem.getInt(offset);\n    offset += Integer.BYTES;\n\n    Util.checkBounds(offset, numBytes, mem.getCapacity());\n    byte[] sortKeyBytes = new byte[numBytes];\n    mem.getByteArray(offset, sortKeyBytes, 0, numBytes);\n    input.setBuffer(sortKeyBytes);\n\n    try {\n      List<SortKey> sortKeys = listSerializer.deserialize(input);\n      SortKey[] array = new SortKey[numItems];\n      sortKeys.toArray(array);\n      input.releaseArrays();\n      return array;\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Failed to deserialize sort key sketch\", e);\n    }\n  }\n\n  @Override\n  public int sizeOf(SortKey item) {\n    return serializeToByteArray(item).length;\n  }\n\n  @Override\n  public int sizeOf(Memory mem, long offset, int numItems) {\n    Preconditions.checkArgument(mem != null, \"Invalid input memory: null\");\n    if (numItems <= 0) {\n      return 0;\n    }\n\n    Util.checkBounds(offset, Integer.BYTES, mem.getCapacity());\n    int numBytes = mem.getInt(offset);\n    return Integer.BYTES + numBytes;","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySketchSerializer.java#L95-L131","documentation":"SortKeySketchSerializer.deserializeFromMemory wraps IOException while reading numItems sort keys back out of the sketch's memory (via the list serializer) into UncheckedIOException \"Failed to deserialize sort key sketch\". It indicates the serialized bytes in memory don't match what the current serializer expects.","triggerScenarios":"Calling deserializeFromMemory (e.g. during sketch deserialization in DataSketches-backed sort-key statistics) when bytes were written by an incompatible serializer version — notably the v1/v2 SortKeySerializer version change.","commonSituations":"Restoring a Flink job (or merging sketches) where statistics bytes were produced with the older v1 SortKeySerializer but are read with the v2 serializer.","solutions":["Use the version-aware path (StatisticsUtil.deserializeCompletedStatistics) that falls back to sort key serializer version 1, rather than calling deserializeFromMemory with the default serializer.","Align the serializer version with the data's version before deserializing.","Discard stale statistics state and rebuild from a fresh run if version provenance is unknown."],"exampleFix":"// before: direct deserialize with latest serializer\nSortKey[] keys = sketchSerializer.deserializeFromMemory(mem, offset, n);\n// after: let StatisticsUtil handle version fallback\nCompletedStatistics stats = StatisticsUtil.deserializeCompletedStatistics(bytes, completedSerializer);","handlingStrategy":"fallback","validationCode":"Preconditions.checkNotNull(mem, \"Invalid input memory: null\");\nPreconditions.checkArgument(numItems >= 0, \"numItems must be non-negative\");","typeGuard":null,"tryCatchPattern":"try {\n  SortKey[] keys = serializer.deserializeFromMemory(mem, offset, n);\n} catch (UncheckedIOException e) {\n  // fall back to v1 serializer or discard stale statistics\n}","preventionTips":["Prefer StatisticsUtil.deserializeCompletedStatistics for version-aware reads","Track which serializer version wrote sketch bytes","Treat statistics state as disposable — it can be rebuilt"],"tags":["flink","deserialization","unchecked-io","version-compatibility"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}