{"record":{"id":"22b1d9804eaaa704","repo":"apache/iceberg","slug":"encoded-string-reached-maximum-length","errorCode":null,"errorMessage":"Encoded string reached maximum length: ","messagePattern":"Encoded string reached maximum length: ","errorType":"exception","errorClass":"UTFDataFormatException","httpStatus":null,"severity":"error","filePath":"flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java","lineNumber":65,"sourceCode":"   *\n   * <p>See * <a href=\"https://issues.apache.org/jira/browse/FLINK-34228\">FLINK-34228</a> * <a\n   * href=\"https://github.com/apache/flink/pull/24191\">https://github.com/apache/flink/pull/24191</a>\n   *\n   * @param out the output stream to write the string to.\n   * @param str the string value to be written.\n   */\n  public static void writeLongUTF(DataOutputView out, String str) throws IOException {\n    int strlen = str.length();\n    long utflen = 0;\n    int ch;\n\n    /* use charAt instead of copying String to char array */\n    for (int i = 0; i < strlen; i++) {\n      ch = str.charAt(i);\n      utflen += getUTFBytesSize(ch);\n\n      if (utflen > Integer.MAX_VALUE) {\n        throw new UTFDataFormatException(\"Encoded string reached maximum length: \" + utflen);\n      }\n    }\n\n    if (utflen > Integer.MAX_VALUE - 4) {\n      throw new UTFDataFormatException(\"Encoded string is too long: \" + utflen);\n    }\n\n    out.writeInt((int) utflen);\n    writeUTFBytes(out, str, (int) utflen);\n  }\n\n  /**\n   * Similar to {@link DataInputDeserializer#readUTF()}. Except this supports larger payloads which\n   * is up to max integer value.\n   *\n   * <p>Note: This method can be removed when the method which does similar thing within the {@link\n   * DataOutputSerializer} already which does the same thing, so use that one instead once that is\n   * released on Flink version 1.20.","sourceCodeStart":47,"sourceCodeEnd":83,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java#L47-L83","documentation":"SerializerHelper.writeLongUTF encodes a String as modified UTF-8 with an explicit length prefix. While accumulating encoded length, if the running total exceeds Integer.MAX_VALUE it throws UTFDataFormatException, because the length cannot be represented in the following 4-byte int prefix. It is a hard size ceiling on any single string serialized this way.","triggerScenarios":"Serializing a single String field via writeLongUTF whose modified-UTF-8 encoding exceeds Integer.MAX_VALUE (~2.1 billion) bytes — effectively only reachable with enormous strings or a length-accumulation overflow.","commonSituations":"Serializing huge VARCHAR payloads or corrupted/attacker-controlled length bookkeeping; streaming unbounded string data through a Flink serializer without chunking.","solutions":["Chunk the string into smaller segments before serializing and reassemble on read","Do not pass unbounded strings through this serializer; enforce an application-level maximum size upstream","If legitimate data approaches 2GB, switch to a length-prefixed long/segmented representation instead of one writeLongUTF call"],"exampleFix":"// before\nhelper.writeLongUTF(hugeString); // > Integer.MAX_VALUE encoded bytes\n// after\nfor (String chunk : splitIntoChunks(hugeString, 1 << 20)) {\n  helper.writeLongUTF(chunk);\n}","handlingStrategy":"validation","validationCode":"static final int MAX_ENCODED = Integer.MAX_VALUE;\nif (estimateModifiedUtf8Length(s) > MAX_ENCODED) {\n  throw new IllegalArgumentException(\"String too large to serialize via writeLongUTF\");\n}\nserializerHelper.writeLongUTF(s);","typeGuard":"boolean canSerialize(String s) { return s != null && estimateModifiedUtf8Length(s) <= Integer.MAX_VALUE; }","tryCatchPattern":"try {\n  helper.writeLongUTF(str);\n} catch (UTFDataFormatException e) {\n  throw new IOException(\"String exceeds serializer limit; chunk or compress the payload\", e);\n}","preventionTips":["Cap application string sizes well below 2GB","Chunk large text fields into segments before serializing","Never stream unbounded user input through a single writeLongUTF call"],"tags":["flink","serialization","utf-8","size-limit"],"backgroundTag":"value-out-of-range","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}