{"record":{"id":"97954ee17fe2d289","repo":"apache/iceberg","slug":"failed-to-serialize-sort-key-97954e","errorCode":null,"errorMessage":"Failed to serialize sort key","messagePattern":"Failed to serialize sort key","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySketchSerializer.java","lineNumber":69,"sourceCode":"  SortKeySketchSerializer(TypeSerializer<SortKey> itemSerializer) {\n    this.itemSerializer = itemSerializer;\n    this.listSerializer = new ListSerializer<>(itemSerializer);\n    this.input = new DataInputDeserializer();\n  }\n\n  @Override\n  public byte[] serializeToByteArray(SortKey item) {\n    try {\n      DataOutputSerializer output = new DataOutputSerializer(DEFAULT_SORT_KEY_SIZE);\n      itemSerializer.serialize(item, output);\n      byte[] itemBytes = output.getSharedBuffer();\n      int numBytes = output.length();\n      byte[] out = new byte[numBytes + Integer.BYTES];\n      ByteArrayUtil.copyBytes(itemBytes, 0, out, 4, numBytes);\n      ByteArrayUtil.putIntLE(out, 0, numBytes);\n      return out;\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Failed to serialize sort key\", e);\n    }\n  }\n\n  @Override\n  public byte[] serializeToByteArray(SortKey[] items) {\n    try {\n      DataOutputSerializer output = new DataOutputSerializer(DEFAULT_SORT_KEY_SIZE * items.length);\n      listSerializer.serialize(Arrays.asList(items), output);\n      byte[] itemsBytes = output.getSharedBuffer();\n      int numBytes = output.length();\n      byte[] out = new byte[Integer.BYTES + numBytes];\n      ByteArrayUtil.putIntLE(out, 0, numBytes);\n      System.arraycopy(itemsBytes, 0, out, Integer.BYTES, numBytes);\n      return out;\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Failed to serialize sort key\", e);\n    }\n  }","sourceCodeStart":51,"sourceCodeEnd":87,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySketchSerializer.java#L51-L87","documentation":"SortKeySketchSerializer.serializeToByteArray(SortKey) wraps any IOException from writing the sort key into an UncheckedIOException with message \"Failed to serialize sort key\". This serializer feeds the reservoir-sketch sampling; a failure here means the underlying TypeSerializer could not encode the SortKey to the output stream.","triggerScenarios":"Called during sketch sampling when sizeOf() or sketch update needs the byte representation and the underlying sort-key serialization throws IOException (e.g., serializer state invalid, incompatible schema after evolution, or I/O failure in the backing output).","commonSituations":"Schema evolution changed a sort-key column type so the cached serializer no longer matches live SortKey objects; null/incoherent SortKey fields; output stream failures in the operator.","solutions":["Verify the sort key's schema matches the serializer's schema — rebuild the serializer after schema changes instead of reusing a stale one.","Check that sort-key columns are not dropped/retyped in an evolved schema before resuming from a savepoint.","Catch UncheckedIOException in sampling code to fail gracefully and skip statistics collection rather than kill the job, if statistics are optional.","Inspect getCause() for the underlying IOException to identify whether it is a schema or stream problem."],"exampleFix":"// before\nbyte[] bytes = sketchSerializer.serializeToByteArray(sortKey);\n// after\ntry {\n  byte[] bytes = sketchSerializer.serializeToByteArray(sortKey);\n} catch (UncheckedIOException e) {\n  LOG.warn(\"Skipping sort key statistics: {}\", e.getCause().getMessage());\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  byte[] bytes = sketchSerializer.serializeToByteArray(sortKey);\n} catch (UncheckedIOException e) {\n  LOG.warn(\"Skipping sort key: {}\", e.getCause().getMessage());\n}","preventionTips":["Rebuild serializers after schema evolution","Verify sort-key column types are unchanged before resuming state","Inspect getCause() for the root IOException"],"tags":["flink","serialization","sketch","unchecked-io"],"backgroundTag":"json-serialization-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}