{"record":{"id":"2fdc5c557aaf3802","repo":"apache/flink","slug":"encoded-string-is-too-long","errorCode":null,"errorMessage":"Encoded string is too long: {}","messagePattern":"Encoded string is too long: (.+?)","errorType":"exception","errorClass":"UTFDataFormatException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/types/Record.java","lineNumber":1743,"sourceCode":"        public void writeUTF(String str) throws IOException {\n            int strlen = str.length();\n            int utflen = 0;\n            int c;\n\n            /* use charAt instead of copying String to char array */\n            for (int i = 0; i < strlen; i++) {\n                c = str.charAt(i);\n                if ((c >= 0x0001) && (c <= 0x007F)) {\n                    utflen++;\n                } else if (c > 0x07FF) {\n                    utflen += 3;\n                } else {\n                    utflen += 2;\n                }\n            }\n\n            if (utflen > 65535) {\n                throw new UTFDataFormatException(\"Encoded string is too long: \" + utflen);\n            } else if (this.position > this.memory.length - utflen - 2) {\n                resize(utflen + 2);\n            }\n\n            byte[] bytearr = this.memory;\n            int count = this.position;\n\n            bytearr[count++] = (byte) ((utflen >>> 8) & 0xFF);\n            bytearr[count++] = (byte) ((utflen >>> 0) & 0xFF);\n\n            int i = 0;\n            for (i = 0; i < strlen; i++) {\n                c = str.charAt(i);\n                if (!((c >= 0x0001) && (c <= 0x007F))) {\n                    break;\n                }\n                bytearr[count++] = (byte) c;\n            }","sourceCodeStart":1725,"sourceCodeEnd":1761,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/types/Record.java#L1725-L1761","documentation":"Thrown by Record's writeUTF implementation, mirroring java.io.DataOutputStream.writeUTF: before writing, it counts the UTF-8 encoded length of the string (1 byte per char in U+0001..U+007F, 2 or 3 bytes otherwise) and rejects strings whose encoding exceeds 65,535 bytes, because the format stores the length in an unsigned 2-byte prefix. This is a UTFDataFormatException, an IOException subtype.","triggerScenarios":"Calling Record's writeUTF (or serializing a Record containing a very large String field) where the modified-UTF-8 encoding of the string exceeds 65,535 bytes — typically long strings, or strings with many non-Latin characters that encode at 2–3 bytes each.","commonSituations":"Packing payloads, JSON blobs, or stack traces into a Record string field; migrating data that previously used a different transport without the 64KB limit; strings that pass length checks in chars but blow the limit in encoded bytes due to non-ASCII content.","solutions":["Shorten or truncate the string so its modified-UTF-8 encoding is <= 65535 bytes","Move large payloads out of the string field: write them via write(byte[]) after writing a length prefix","If you control the schema, split the field or compress the payload before serialization"],"exampleFix":"// before\nrecord.setField(0, hugeJsonString); // > 64KB encoded\n\n// after\nbyte[] payload = compress(hugeJsonString.getBytes(StandardCharsets.UTF_8));\nrecord.setField(0, Base64.getEncoder().encodeToString(payload));","handlingStrategy":"validation","validationCode":"static boolean fitsModifiedUtf8(String s) {\n    long utflen = 0;\n    for (int i = 0; i < s.length(); i++) {\n        char c = s.charAt(i);\n        utflen += (c >= 0x0001 && c <= 0x007F) ? 1 : (c > 0x07FF ? 3 : 2);\n    }\n    return utflen <= 65535;\n}","typeGuard":null,"tryCatchPattern":"catch (UTFDataFormatException e) { // truncate or offload the payload }","preventionTips":["Keep Record string fields well under 64KB encoded","Account for 2-3 bytes per non-ASCII char","Offload large text to byte[] fields or external storage"],"tags":["flink","serialization","record","utf","size-limit"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}