apache/iceberg · error · UTFDataFormatException

Encoded string is too long:

Error message

Encoded string is too long: 

What it means

After computing total encoded length, writeLongUTF requires room for a 4-byte int length header: if utflen exceeds Integer.MAX_VALUE - 4 it throws UTFDataFormatException('Encoded string is too long'). It is the final, stricter guard before writing the int-prefixed modified-UTF payload.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java:70

   * @param str the string value to be written.
   */
  public static void writeLongUTF(DataOutputView out, String str) throws IOException {
    int strlen = str.length();
    long utflen = 0;
    int ch;

    /* use charAt instead of copying String to char array */
    for (int i = 0; i < strlen; i++) {
      ch = str.charAt(i);
      utflen += getUTFBytesSize(ch);

      if (utflen > Integer.MAX_VALUE) {
        throw new UTFDataFormatException("Encoded string reached maximum length: " + utflen);
      }
    }

    if (utflen > Integer.MAX_VALUE - 4) {
      throw new UTFDataFormatException("Encoded string is too long: " + utflen);
    }

    out.writeInt((int) utflen);
    writeUTFBytes(out, str, (int) utflen);
  }

  /**
   * Similar to {@link DataInputDeserializer#readUTF()}. Except this supports larger payloads which
   * is up to max integer value.
   *
   * <p>Note: This method can be removed when the method which does similar thing within the {@link
   * DataOutputSerializer} already which does the same thing, so use that one instead once that is
   * released on Flink version 1.20.
   *
   * <p>See * <a href="https://issues.apache.org/jira/browse/FLINK-34228">FLINK-34228</a> * <a
   * href="https://github.com/apache/flink/pull/24191">https://github.com/apache/flink/pull/24191</a>
   *
   * @param in the input stream to read the string from.

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Reduce the string size below Integer.MAX_VALUE - 4 encoded bytes
2. Chunk or compress the payload before writing
3. Persist large values via external storage and write a reference

Example fix

// before
helper.writeLongUTF(out, str); // may exceed Integer.MAX_VALUE - 4
// after
int size = SerializerHelper.getUTFBytesSize(str); // or compute helper-style
if (size > Integer.MAX_VALUE - 4) {
  throw new IllegalArgumentException("String too large for longUTF; chunk or compress");
}
helper.writeLongUTF(out, str);

int encoded = str.getBytes(StandardCharsets.UTF_8).length; // approximate
if (encoded >= Integer.MAX_VALUE - 4) throw new IllegalArgumentException("String too long for writeLongUTF");

static boolean fitsLongUTF(String s) {
  return s == null || s.getBytes(StandardCharsets.UTF_8).length < Integer.MAX_VALUE - 4;
}

try { helper.writeLongUTF(out, str); }
catch (UTFDataFormatException e) {
  if (e.getMessage().contains("too long")) { compressAndWrite(out, str); }
  else throw e;
}

When it happens

Trigger: Writing a string whose UTF-8 size is between Integer.MAX_VALUE-3 and Integer.MAX_VALUE bytes via SerializerHelper.writeLongUTF.

Common situations: Same as the maximum-length guard: oversized single-string payloads in serializer pipelines; edge-case sizes just under 2GB that still fail because of the length header.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• Encoded string reached maximum length:
• Encoded string reached maximum length: {utflen}
• Encoded string is too long: {utflen}
• malformed input: partial character at end
• malformed input around byte {count}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/a83747744db79250. Report an issue: GitHub.