{"record":{"id":"7e6450a930493130","repo":"apache/hadoop","slug":"string-was-too-long-to-write-expected-less-than","errorCode":null,"errorMessage":"string was too long to write!  Expected less than or equal to {} bytes, but got {} bytes.","messagePattern":"string was too long to write!  Expected less than or equal to (.+?) bytes, but got (.+?) bytes\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java","lineNumber":603,"sourceCode":"    WritableUtils.writeVInt(out, length);\n    out.write(bytes.array(), 0, length);\n    return length;\n  }\n\n  /**\n   * @return Write a UTF8 encoded string with a maximum size to out.\n   *\n   * @param out input out.\n   * @param s input s.\n   * @param maxLength input maxLength.\n   * @throws IOException raised on errors performing I/O.\n   */\n  public static int writeString(DataOutput out, String s, int maxLength)\n      throws IOException {\n    ByteBuffer bytes = encode(s);\n    int length = bytes.limit();\n    if (length > maxLength) {\n      throw new IOException(\"string was too long to write!  Expected \" +\n          \"less than or equal to \" + maxLength + \" bytes, but got \" +\n          length + \" bytes.\");\n    }\n    WritableUtils.writeVInt(out, length);\n    out.write(bytes.array(), 0, length);\n    return length;\n  }\n\n  ////// states for validateUTF8\n\n  private static final int LEAD_BYTE = 0;\n\n  private static final int TRAIL_BYTE_1 = 1;\n\n  private static final int TRAIL_BYTE = 2;\n\n  /**\n   * Check if a byte array contains valid UTF-8.","sourceCodeStart":585,"sourceCodeEnd":621,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java#L585-L621","documentation":"Text.writeString(DataOutput, String, int maxLength) encodes the string to UTF-8 and refuses to write more than maxLength bytes, throwing IOException(\"string was too long to write! Expected less than or equal to M bytes, but got N bytes.\"). The limit is on encoded bytes, not characters, and the method returns the encoded length on success.","triggerScenarios":"Serializing a string whose UTF-8 byte length exceeds the limit the format/reader imposes; strings that look short in characters but expand 3-4x in UTF-8 (CJK, emoji); raising the reader limit without raising the writer's maxLength (or vice versa).","commonSituations":"Config keys/values, names, or identifiers written into length-bounded slots; internationalized content blowing past limits sized for ASCII; mismatched caps after a limit change.","solutions":["Check the encoded size before writing: int len = Text.encode(s).limit(); then reject or truncate deliberately.","Raise maxLength on both write and read paths if larger values are legitimate.","Test limits with multi-byte content (3-byte CJK, 4-byte emoji), not just ASCII."],"exampleFix":"// before\nText.writeString(out, s, 256);           // throws on long/multibyte strings\n\n// after: fail with an actionable error at the call site\nByteBuffer b = Text.encode(s);\nif (b.limit() > MAX_LEN) {\n  throw new IllegalArgumentException(\"value is \" + b.limit()\n      + \" UTF-8 bytes, limit \" + MAX_LEN);\n}\nText.writeString(out, s, MAX_LEN);","handlingStrategy":"validation","validationCode":"ByteBuffer b = Text.encode(s);\nif (b.limit() > maxLength) {\n  throw new IllegalArgumentException(\n      \"String encodes to \" + b.limit() + \" UTF-8 bytes, limit \" + maxLength);\n}\nText.writeString(out, s, maxLength);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Measure limits in UTF-8 bytes (Text.encode(s).limit()), never in characters","Test with CJK and emoji content when setting string limits","Share one limit constant between writeString and the corresponding read"],"tags":["text","string","utf-8","size-limit","serialization","hadoop"],"backgroundTag":"payload-too-large","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}