{"record":{"id":"166617735c5fb193","repo":"apache/iceberg","slug":"malformed-input-partial-character-at-end-166617","errorCode":null,"errorMessage":"malformed input: partial character at end","messagePattern":"malformed input: partial character at end","errorType":"exception","errorClass":"UTFDataFormatException","httpStatus":null,"severity":"error","filePath":"flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java","lineNumber":134,"sourceCode":"      switch (ch >> 4) {\n        case 0:\n        case 1:\n        case 2:\n        case 3:\n        case 4:\n        case 5:\n        case 6:\n        case 7:\n          /* 0xxxxxxx */\n          count++;\n          chararr[chararrCount++] = (char) ch;\n          break;\n        case 12:\n        case 13:\n          /* 110x xxxx 10xx xxxx */\n          count += 2;\n          if (count > utflen) {\n            throw new UTFDataFormatException(\"malformed input: partial character at end\");\n          }\n          char2 = bytearr[count - 1];\n          if ((char2 & 0xC0) != 0x80) {\n            throw new UTFDataFormatException(\"malformed input around byte \" + count);\n          }\n          chararr[chararrCount++] = (char) (((ch & 0x1F) << 6) | (char2 & 0x3F));\n          break;\n        case 14:\n          /* 1110 xxxx 10xx xxxx 10xx xxxx */\n          count += 3;\n          if (count > utflen) {\n            throw new UTFDataFormatException(\"malformed input: partial character at end\");\n          }\n          char2 = bytearr[count - 2];\n          char3 = bytearr[count - 1];\n          if (((char2 & 0xC0) != 0x80) || ((char3 & 0xC0) != 0x80)) {\n            throw new UTFDataFormatException(\"malformed input around byte \" + (count - 1));\n          }","sourceCodeStart":116,"sourceCodeEnd":152,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java#L116-L152","documentation":"readLongUTF validates that every declared 2-byte (110xxxxx) character fits entirely within the declared UTF length; when the final byte boundary cuts a character in half (count > utflen) it throws UTFDataFormatException('malformed input: partial character at end'). This means the incoming byte stream is truncated or its declared length does not match its content.","triggerScenarios":"Deserializing a payload where the declared int length prefix is larger than the actual encoded bytes, or the stream was truncated mid-character (network cut, partial write, corrupted file).","commonSituations":"Kafka/serialization payloads cut off by buffer limits; mismatched writer/reader versions producing inconsistent length headers; corruption when copying serialized snapshots.","solutions":["Validate the producer wrote the correct length header and the full byte array (check writer/reader version parity)","Re-read or re-fetch the payload — check for truncation in the transport (Kafka message size limits, socket timeouts)","Catch UTFDataFormatException on read and log the byte offsets to isolate the corrupted record, then re-serialize the affected data"],"exampleFix":"// before\nString s = SerializerHelper.readLongUTF(in); // throws on truncated input\n// after\ntry {\n  String s = SerializerHelper.readLongUTF(in);\n} catch (UTFDataFormatException e) {\n  LOG.warn(\"Truncated/corrupted UTF payload, re-fetching record\", e);\n  s = fallbackRead();\n}","handlingStrategy":"try-catch","validationCode":"// before reading: verify buffer holds utflen bytes after the int header\nif (buf.position() + declaredLen > buf.limit()) {\n  throw new EOFException(\"Declared UTF length exceeds available bytes\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  String s = SerializerHelper.readLongUTF(in);\n} catch (UTFDataFormatException e) {\n  if (e.getMessage().contains(\"partial character at end\") || e.getMessage().contains(\"malformed input\")) {\n    log.warn(\"Truncated/corrupt UTF record at offset {}\", bytesConsumed, e);\n    s = null; // re-fetch or skip record\n  } else throw e;\n}","preventionTips":["Ensure writers always write the complete payload after the length header","Check transport limits (Kafka max.message.bytes, socket buffers) for truncation","Keep writer/reader SerializerHelper versions identical","Checksum large serialized payloads to detect corruption early"],"tags":["flink","serialization","utf","corruption"],"backgroundTag":"invalid-argument-format","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}