apache/flink · critical · UTFDataFormatException

Encoded string reached maximum length: {utflen}

Error message

Encoded string reached maximum length: {utflen}

What it means

Thrown by DataOutputSerializer.writeLongUTF(String) during the per-character byte-size accumulation when the running total (a long) exceeds Integer.MAX_VALUE. Because the backing buffer is a Java array (int-indexed), the encoded length cannot exceed ~2GB, so writeLongUTF aborts as soon as the running total crosses that threshold.

Source

Thrown at flink-core/src/main/java/org/apache/flink/core/memory/DataOutputSerializer.java:276

    /**
     * Similar to {@link #writeUTF(String)}. The size is only limited by the maximum java array size
     * of the buffer.
     *
     * @param str the string value to be written.
     * @throws IOException if an I/O error occurs.
     */
    public void writeLongUTF(String str) throws IOException {
        int strlen = str.length();
        long utflen = 0;
        int c;

        /* use charAt instead of copying String to char array */
        for (int i = 0; i < strlen; i++) {
            c = str.charAt(i);
            utflen += getUTFBytesSize(c);

            if (utflen > Integer.MAX_VALUE) {
                throw new UTFDataFormatException(
                        "Encoded string reached maximum length: " + utflen);
            }
        }

        if (utflen > Integer.MAX_VALUE - 4) {
            throw new UTFDataFormatException("Encoded string is too long: " + utflen);
        } else if (this.position > this.buffer.length - utflen - 2) {
            resize((int) utflen + 4);
        }

        writeInt((int) utflen);

        writeUTFBytes(str);
    }

    private void writeUTFBytes(String str) {
        int strlen = str.length();
        int c;

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Avoid holding such large payloads as a single String; serialize as a byte[] or stream instead.
2. Enforce a maximum input string length at the API boundary before serialization.
3. If genuinely large binary data is needed, write it as raw bytes with an int length prefix rather than a modified-UTF-8 String.

static long modifiedUtf8LenLong(String s) {
    long n = 0;
    for (int i = 0; i < s.length(); i++) {
        int c = s.charAt(i);
        n += (c >= 0x0001 && c <= 0x007F) ? 1 : (c > 0x07FF ? 3 : 2);
    }
    return n;
}
// if (modifiedUtf8LenLong(str) > Integer.MAX_VALUE) reject or serialize as raw bytes

When it happens

Trigger: Calling writeLongUTF(str) on a string whose modified-UTF-8 encoding would exceed Integer.MAX_VALUE (~2.1GB) bytes.

Common situations: Pathologically large in-memory strings; concatenating unbounded user/audit content into a single string; a memory-resident blob mistakenly treated as a String.

Related errors

• Encoded string is too long: {utflen}
• encoded string too long: {} memory
• Serialization failed because the record length would exceed
• malformed input: partial character at end
• malformed input around byte {}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/ebf76ffa26f4790e. Report an issue: GitHub.