{"record":{"id":"ba133ebbb2f577bc","repo":"apache/flink","slug":"charsequence-is-too-long","errorCode":null,"errorMessage":"CharSequence is too long.","messagePattern":"CharSequence is too long\\.","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/types/StringValue.java","lineNumber":806,"sourceCode":"                    c |= (curr & 0x7f) << shift;\n                    shift += 7;\n                }\n                c |= curr << shift;\n            }\n            data[i] = (char) c;\n        }\n\n        return new String(data, 0, len);\n    }\n\n    public static final void writeString(CharSequence cs, DataOutput out) throws IOException {\n        if (cs != null) {\n            int strlen = cs.length();\n\n            // the length we write is offset by one, because a length of zero indicates a null value\n            int lenToWrite = strlen + 1;\n            if (lenToWrite < 0) {\n                throw new IllegalArgumentException(\"CharSequence is too long.\");\n            }\n\n            // string is prefixed by it's variable length encoded size, which can take 1-5 bytes.\n            if (lenToWrite < HIGH_BIT) {\n                out.write((byte) lenToWrite);\n            } else if (lenToWrite < HIGH_BIT14) {\n                out.write((lenToWrite | HIGH_BIT));\n                out.write((lenToWrite >>> 7));\n            } else if (lenToWrite < HIGH_BIT21) {\n                out.write(lenToWrite | HIGH_BIT);\n                out.write((lenToWrite >>> 7) | HIGH_BIT);\n                out.write((lenToWrite >>> 14));\n            } else if (lenToWrite < HIGH_BIT28) {\n                out.write(lenToWrite | HIGH_BIT);\n                out.write((lenToWrite >>> 7) | HIGH_BIT);\n                out.write((lenToWrite >>> 14) | HIGH_BIT);\n                out.write((lenToWrite >>> 21));\n            } else {","sourceCodeStart":788,"sourceCodeEnd":824,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/types/StringValue.java#L788-L824","documentation":"Thrown by StringValue.writeString(CharSequence, DataOutput) when the character sequence length overflows the int used for the length header. The wire format stores strlen + 1 (zero is reserved for null), so a sequence of length Integer.MAX_VALUE makes lenToWrite wrap to a negative int. This is a hard limit of Flink's variable-length string encoding: a single CharSequence of ~2^31 chars cannot be serialized with this method.","triggerScenarios":"Calling StringValue.writeString(cs, out) (directly or via StringValue.write()/TypeSerializer paths that delegate to it) with a CharSequence whose length() >= Integer.MAX_VALUE; lenToWrite = strlen + 1 becomes negative and the guard at StringValue.java:805 fires.","commonSituations":"Building giant in-memory strings/buffers (e.g. concatenating large datasets or base64 blobs into one field), reading a huge file into a single String, or a buggy upstream producing an unbounded CharSequence. Realistically hit only with multi-GB character arrays (2GB of chars = 4GB heap).","solutions":["Investigate why a single field is near 2^31 characters; it is almost always a bug (unbounded concatenation or wrong field boundaries) rather than legitimate data.","Split the payload into multiple records/fields or stream it through a different representation (byte[], or a file/source split) instead of one CharSequence.","Pre-check cs.length() < Integer.MAX_VALUE before calling writeString and fail with a domain-specific error naming the offending field.","If huge single values are genuinely required, serialize with a format without the int-length limit rather than this legacy encoding."],"exampleFix":"// before\nStringValue.writeString(hugeCharSequence, dataOutput); // may overflow length header\n\n// after\nif (hugeCharSequence.length() >= Integer.MAX_VALUE) {\n    throw new IllegalArgumentException(\n        \"Field too large for StringValue encoding: \" + hugeCharSequence.length() + \" chars\");\n}\nStringValue.writeString(hugeCharSequence, dataOutput);","handlingStrategy":"validation","validationCode":"if (cs == null || cs.length() >= Integer.MAX_VALUE) {\n    throw new IllegalArgumentException(\n        \"CharSequence too long for StringValue encoding: \" + (cs == null ? -1 : cs.length()));\n}","typeGuard":null,"tryCatchPattern":"try {\n    StringValue.writeString(cs, out);\n} catch (IllegalArgumentException e) {\n    if (!\"CharSequence is too long.\".equals(e.getMessage())) throw e;\n    // split payload or fail with domain-specific error\n}","preventionTips":["Never accumulate unbounded content into a single CharSequence field; cap sizes at ingestion.","Assert cs.length() < Integer.MAX_VALUE before serialization in hot paths that aggregate strings.","Prefer streaming large payloads through files/source splits instead of one giant field."],"tags":["serialization","stringvalue","overflow","flink-core"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}