apache/iceberg · error · UTFDataFormatException

malformed input: partial character at end

Error message

malformed input: partial character at end

What it means

SerializerHelper.readLongUTF decodes modified UTF-8 byte-by-byte. A lead byte of 110xxxxx (cases 12-13) announces a 2-byte sequence; if the stream ends before the continuation byte arrives (count > utflen), the input is truncated and cannot be decoded. The method throws UTFDataFormatException because the encoded data is corrupt or cut off.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java:134

      switch (ch >> 4) {
        case 0:
        case 1:
        case 2:
        case 3:
        case 4:
        case 5:
        case 6:
        case 7:
          /* 0xxxxxxx */
          count++;
          chararr[chararrCount++] = (char) ch;
          break;
        case 12:
        case 13:
          /* 110x xxxx 10xx xxxx */
          count += 2;
          if (count > utflen) {
            throw new UTFDataFormatException("malformed input: partial character at end");
          }
          char2 = bytearr[count - 1];
          if ((char2 & 0xC0) != 0x80) {
            throw new UTFDataFormatException("malformed input around byte " + count);
          }
          chararr[chararrCount++] = (char) (((ch & 0x1F) << 6) | (char2 & 0x3F));
          break;
        case 14:
          /* 1110 xxxx 10xx xxxx 10xx xxxx */
          count += 3;
          if (count > utflen) {
            throw new UTFDataFormatException("malformed input: partial character at end");
          }
          char2 = bytearr[count - 2];
          char3 = bytearr[count - 1];
          if (((char2 & 0xC0) != 0x80) || ((char3 & 0xC0) != 0x80)) {
            throw new UTFDataFormatException("malformed input around byte " + (count - 1));
          }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Regenerate/re-read the data from a trusted source — the stream bytes are corrupt
2. Verify the writer and reader use the same SerializerHelper/DataOutput encoding
3. Check for truncation upstream (partial writes, short reads, wrong length prefix)
4. Validate byte payload integrity (checksums) before deserializing

try {
  String s = SerializerHelper.readLongUTF(in);
} catch (UTFDataFormatException e) {
  // treat stream as corrupt: fail the job or restore from a valid checkpoint
  throw new IOException("corrupt UTF stream, restore from checkpoint", e);
}

When it happens

Trigger: Reading a stream whose byte length header (utflen) ends mid-character — i.e. a multi-byte UTF-8 sequence split across the declared end of data.

Common situations: Corrupted Flink checkpoint/state files; truncated network or file streams; bytes written by a producer using a different UTF-8 variant or wrong length header.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• malformed input: partial character at end
• malformed input: partial character at end
• malformed input around byte
• Encoded string reached maximum length: {utflen}
• Encoded string is too long: {utflen}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/a6f9878254bbe063. Report an issue: GitHub.