apache/iceberg · error · UTFDataFormatException

malformed input: partial character at end

Error message

malformed input: partial character at end

What it means

readLongUTF decodes a modified-UTF-8 byte stream. When a 2-byte lead byte (0xC0-0xDF) signals a continuation but the stream ends before the second byte, the decoder throws UTFDataFormatException 'malformed input: partial character at end'. The byte array is truncated or corrupt.

Source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java:134

      switch (ch >> 4) {
        case 0:
        case 1:
        case 2:
        case 3:
        case 4:
        case 5:
        case 6:
        case 7:
          /* 0xxxxxxx */
          count++;
          chararr[chararrCount++] = (char) ch;
          break;
        case 12:
        case 13:
          /* 110x xxxx 10xx xxxx */
          count += 2;
          if (count > utflen) {
            throw new UTFDataFormatException("malformed input: partial character at end");
          }
          char2 = bytearr[count - 1];
          if ((char2 & 0xC0) != 0x80) {
            throw new UTFDataFormatException("malformed input around byte " + count);
          }
          chararr[chararrCount++] = (char) (((ch & 0x1F) << 6) | (char2 & 0x3F));
          break;
        case 14:
          /* 1110 xxxx 10xx xxxx 10xx xxxx */
          count += 3;
          if (count > utflen) {
            throw new UTFDataFormatException("malformed input: partial character at end");
          }
          char2 = bytearr[count - 2];
          char3 = bytearr[count - 1];
          if (((char2 & 0xC0) != 0x80) || ((char3 & 0xC0) != 0x80)) {
            throw new UTFDataFormatException("malformed input around byte " + (count - 1));
          }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Verify the producer wrote with the same SerializerHelper.writeLongUTF and that the payload is intact.
2. Check byte array length vs the written utflen prefix before decoding.
3. Re-read the source data / re-run the job; if persistent, suspect corruption or a serialization version mismatch.

Example fix

// before
String s = helper.readLongUTF(in);
// after
int expected = in.readInt();
byte[] buf = new byte[expected];
int n = in.readFully(buf);
if (n != expected) { throw new IOException("truncated payload"); }

// java
int len = readLengthPrefix(buf);
if (buf.remaining() < len) { throw new IOException("truncated payload: need " + len + " bytes"); }

boolean isCompleteUtfPayload(byte[] bytes) {
  // last lead byte must not be a bare multi-byte lead (0xC0-0xF7)
  if (bytes.length == 0) return true;
  int lead = bytes[bytes.length - 1] & 0xFF;
  return lead < 0x80 || (lead >= 0xC0 && bytes.length >= 2);
}

try {
  String s = helper.readLongUTF(in);
} catch (UTFDataFormatException e) {
  if (e.getMessage().contains("partial character at end")) {
    // re-read source or mark payload corrupt
  } else { throw e; }
}

When it happens

Trigger: Reading a byte[] whose length-prefix (utflen) is larger than the actual trailing bytes available, or the array was truncated mid multi-byte sequence.

Common situations: Corrupt or truncated serialized payloads (network cut, partial file write); hand-crafted byte arrays not produced by writeLongUTF; version-incompatible encoding of the length prefix.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• malformed input: partial character at end
• malformed input: partial character at end
• malformed input around byte
• Encoded string reached maximum length: {utflen}
• Encoded string is too long: {utflen}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/8aef78da5a539da5. Report an issue: GitHub.