apache/flink · error · UTFDataFormatException

Encoded string is too long: {utflen}

Error message

Encoded string is too long: {utflen}

What it means

Thrown by DataOutputSerializer.writeUTF(String) when the modified-UTF-8 encoded byte length of the string exceeds 65535. writeUTF prefixes the string with its byte length as an unsigned 16-bit value, so the encoded form is capped at 64KB. This mirrors java.io.DataOutput.writeUTF limits.

Source

Thrown at flink-core/src/main/java/org/apache/flink/core/memory/DataOutputSerializer.java:245

        }
        this.buffer[this.position++] = (byte) ((v >>> 8) & 0xff);
        this.buffer[this.position++] = (byte) (v & 0xff);
    }

    @Override
    public void writeUTF(String str) throws IOException {
        int strlen = str.length();
        int utflen = 0;
        int c;

        /* use charAt instead of copying String to char array */
        for (int i = 0; i < strlen; i++) {
            c = str.charAt(i);
            utflen += getUTFBytesSize(c);
        }

        if (utflen > 65535) {
            throw new UTFDataFormatException("Encoded string is too long: " + utflen);
        } else if (this.position > this.buffer.length - utflen - 2) {
            resize(utflen + 2);
        }

        byte[] bytearr = this.buffer;

        bytearr[this.position++] = (byte) ((utflen >>> 8) & 0xFF);
        bytearr[this.position++] = (byte) (utflen & 0xFF);

        writeUTFBytes(str);
    }

    /**
     * Similar to {@link #writeUTF(String)}. The size is only limited by the maximum java array size
     * of the buffer.
     *
     * @param str the string value to be written.
     * @throws IOException if an I/O error occurs.

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Switch to writeLongUTF/readLongUTF, which uses a 4-byte (int) length prefix supporting up to ~2GB.
2. Pre-check the encoded length and reject or chunk oversized strings before writeUTF.
3. For unbounded text, use a byte-array based serializer with an int length prefix instead of writeUTF.

Example fix

// before
out.writeUTF(jsonBlob);

// after
out.writeLongUTF(jsonBlob); // paired with input.readLongUTF()

static int modifiedUtf8Len(String s) {
    int n = 0;
    for (int i = 0; i < s.length(); i++) {
        int c = s.charAt(i);
        n += (c >= 0x0001 && c <= 0x007F) ? 1 : (c > 0x07FF ? 3 : 2);
    }
    return n;
}
// if (modifiedUtf8Len(str) > 65535) use writeLongUTF instead

try {
    out.writeUTF(str);
} catch (UTFDataFormatException e) {
    out.writeLongUTF(str); // fallback for strings over 64KB
}

When it happens

Trigger: Calling writeUTF(str) where the encoded UTF-8 byte length > 65535. ASCII strings > 65535 chars trigger this directly; strings with multi-byte chars trigger at correspondingly fewer characters.

Common situations: Serializing large text fields, JSON/XML blobs, stack traces, or long log messages with writeUTF; user-generated content that exceeds 64KB encoded.

Related errors

• Encoded string reached maximum length: {utflen}
• encoded string too long: {} memory
• malformed input: partial character at end
• malformed input around byte {count}
• malformed input around byte {count-1}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/2439ae7a290d7003. Report an issue: GitHub.