{"record":{"id":"4cd068c78d1f71b7","repo":"apache/iceberg","slug":"failed-to-deserialize-sort-key-sketch-4cd068","errorCode":null,"errorMessage":"Failed to deserialize sort key sketch","messagePattern":"Failed to deserialize sort key sketch","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySketchSerializer.java","lineNumber":113,"sourceCode":"\n    long offset = startingOffset;\n    Util.checkBounds(offset, Integer.BYTES, mem.getCapacity());\n    int numBytes = mem.getInt(offset);\n    offset += Integer.BYTES;\n\n    Util.checkBounds(offset, numBytes, mem.getCapacity());\n    byte[] sortKeyBytes = new byte[numBytes];\n    mem.getByteArray(offset, sortKeyBytes, 0, numBytes);\n    input.setBuffer(sortKeyBytes);\n\n    try {\n      List<SortKey> sortKeys = listSerializer.deserialize(input);\n      SortKey[] array = new SortKey[numItems];\n      sortKeys.toArray(array);\n      input.releaseArrays();\n      return array;\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Failed to deserialize sort key sketch\", e);\n    }\n  }\n\n  @Override\n  public int sizeOf(SortKey item) {\n    return serializeToByteArray(item).length;\n  }\n\n  @Override\n  public int sizeOf(Memory mem, long offset, int numItems) {\n    Preconditions.checkArgument(mem != null, \"Invalid input memory: null\");\n    if (numItems <= 0) {\n      return 0;\n    }\n\n    Util.checkBounds(offset, Integer.BYTES, mem.getCapacity());\n    int numBytes = mem.getInt(offset);\n    return Integer.BYTES + numBytes;","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySketchSerializer.java#L95-L131","documentation":"SortKeySketchSerializer.deserializeFromMemory() wraps IOException from decoding sort keys out of the sketch's memory segment into an UncheckedIOException with message \"Failed to deserialize sort key sketch\". This happens when reading a reservoir sketch's sampled items and the byte layout or schema no longer matches the serializer.","triggerScenarios":"Deserializing sampled SortKey items from sketch Memory during sketch merge/read (restore from state, coordinator aggregation) when the item bytes were written with a different schema/sort order or are truncated/corrupted.","commonSituations":"Restoring sketches from a checkpoint after the table's sort order or field types changed; merging sketches produced by operators with mismatched serializer versions; corrupted state bytes.","solutions":["Ensure all subtasks serialize sketches with the same schema and iceberg-flink-runtime version before merge.","Do not change the sort order between checkpoint and restore; take a new savepoint after schema evolution.","Catch UncheckedIOException around sketch deserialization and fall back to discarding stale statistics for one cycle.","Inspect getCause() to confirm whether it is a schema incompatibility or byte corruption."],"exampleFix":"// before\nSortKey[] keys = sketchSerializer.deserializeFromMemory(mem, offset, numItems);\n// after\ntry {\n  SortKey[] keys = sketchSerializer.deserializeFromMemory(mem, offset, numItems);\n} catch (UncheckedIOException e) {\n  LOG.warn(\"Discarding incompatible sketch statistics\", e);\n}","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  SortKey[] keys = sketchSerializer.deserializeFromMemory(mem, offset, numItems);\n} catch (UncheckedIOException e) {\n  LOG.warn(\"Discarding incompatible sketch statistics\", e);\n  return new SortKey[0]; // fall back to no statistics for this cycle\n}","preventionTips":["Freeze sort order while sketch state is live","Use identical runtime versions across all subtasks","Fall back gracefully — statistics are advisory, not correctness-critical"],"tags":["flink","deserialization","sketch","unchecked-io"],"backgroundTag":"protobuf-unmarshal-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}