apache/iceberg · error · UTFDataFormatException

Encoded string is too long: {utflen}

Error message

Encoded string is too long: {utflen}

What it means

SerializerHelper.writeLongUTF reserves 4 bytes for the length header, so the encoded length must fit in an int while leaving room for the header. Strings longer than Integer.MAX_VALUE - 4 encoded UTF-8 bytes are rejected with this error, distinguishing it from the hard-overflow check inside the loop.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java:70

   * @param str the string value to be written.
   */
  public static void writeLongUTF(DataOutputView out, String str) throws IOException {
    int strlen = str.length();
    long utflen = 0;
    int ch;

    /* use charAt instead of copying String to char array */
    for (int i = 0; i < strlen; i++) {
      ch = str.charAt(i);
      utflen += getUTFBytesSize(ch);

      if (utflen > Integer.MAX_VALUE) {
        throw new UTFDataFormatException("Encoded string reached maximum length: " + utflen);
      }
    }

    if (utflen > Integer.MAX_VALUE - 4) {
      throw new UTFDataFormatException("Encoded string is too long: " + utflen);
    }

    out.writeInt((int) utflen);
    writeUTFBytes(out, str, (int) utflen);
  }

  /**
   * Similar to {@link DataInputDeserializer#readUTF()}. Except this supports larger payloads which
   * is up to max integer value.
   *
   * <p>Note: This method can be removed when the method which does similar thing within the {@link
   * DataOutputSerializer} already which does the same thing, so use that one instead once that is
   * released on Flink version 1.20.
   *
   * <p>See * <a href="https://issues.apache.org/jira/browse/FLINK-34228">FLINK-34228</a> * <a
   * href="https://github.com/apache/flink/pull/24191">https://github.com/apache/flink/pull/24191</a>
   *
   * @param in the input stream to read the string from.

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Keep encoded strings comfortably below Integer.MAX_VALUE - 4 bytes (e.g. cap at 2GB minus headroom)
2. Chunk the payload before writing
3. Check length with getUTFBytesSize before calling writeLongUTF

Example fix

// before
helper.writeLongUTF(str);
// after
if (SerializerHelper.getUTFBytesSize(str) <= Integer.MAX_VALUE - 4) {
  helper.writeLongUTF(str);
} else {
  throw new IllegalArgumentException("string too large");
}

if (SerializerHelper.getUTFBytesSize(str) > Integer.MAX_VALUE - 4) {
  throw new IllegalArgumentException("encoded string too large for writeLongUTF");
}

try {
  helper.writeLongUTF(str);
} catch (UTFDataFormatException e) {
  throw new IllegalArgumentException("string exceeds 2GB encoder limit", e);
}

When it happens

Trigger: Calling writeLongUTF with a string whose total encoded length is in the narrow window (Integer.MAX_VALUE - 4, Integer.MAX_VALUE] bytes.

Common situations: Serializing strings very close to the 2GB boundary — rare, usually from concatenated or accumulated payloads in Flink state serialization.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Encoded string reached maximum length: {utflen}
• malformed input: partial character at end
• malformed input around byte {count}
• malformed input around byte
• Encoded string reached maximum length:

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/181a162bbe3bbedf. Report an issue: GitHub.