apache/iceberg · error · UTFDataFormatException

malformed input around byte

Error message

malformed input around byte 

What it means

When decoding a 3-byte modified UTF-8 sequence in readLongUTF, both continuation bytes must match the 10xxxxxx pattern. If either byte fails the check, the decoder throws UTFDataFormatException reporting the byte offset (count - 1). Note the message ends with the offset number appended at runtime.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java:151

          if (count > utflen) {
            throw new UTFDataFormatException("malformed input: partial character at end");
          }
          char2 = bytearr[count - 1];
          if ((char2 & 0xC0) != 0x80) {
            throw new UTFDataFormatException("malformed input around byte " + count);
          }
          chararr[chararrCount++] = (char) (((ch & 0x1F) << 6) | (char2 & 0x3F));
          break;
        case 14:
          /* 1110 xxxx 10xx xxxx 10xx xxxx */
          count += 3;
          if (count > utflen) {
            throw new UTFDataFormatException("malformed input: partial character at end");
          }
          char2 = bytearr[count - 2];
          char3 = bytearr[count - 1];
          if (((char2 & 0xC0) != 0x80) || ((char3 & 0xC0) != 0x80)) {
            throw new UTFDataFormatException("malformed input around byte " + (count - 1));
          }
          chararr[chararrCount++] =
              (char) (((ch & 0x0F) << 12) | ((char2 & 0x3F) << 6) | (char3 & 0x3F));
          break;
        default:
          /* 10xx xxxx, 1111 xxxx */
          throw new UTFDataFormatException("malformed input around byte " + count);
      }
    }
    // The number of chars produced may be less than utflen
    return new String(chararr, 0, chararrCount);
  }

  private static int getUTFBytesSize(int ch) {
    if ((ch >= 0x0001) && (ch <= 0x007F)) {
      return 1;
    } else if (ch > 0x07FF) {
      return 3;

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Verify writer and reader share the exact same modified-UTF-8 encoding
2. Use byte[] fields instead of strings for binary data
3. Validate payload encoding before serialization
4. Restore from a valid checkpoint

try {
  return SerializerHelper.readLongUTF(in);
} catch (UTFDataFormatException e) {
  throw new IOException("malformed UTF sequence: " + e.getMessage(), e);
}

When it happens

Trigger: Reading a stream where byte 2 or 3 of a 3-byte sequence is not a valid continuation byte — invalid encoded data from an incompatible or corrupt source.

Common situations: Binary data passed as strings; bytes written by a different UTF encoding (e.g. standard UTF-8 with 4-byte sequences, CESU-8 mismatch); corrupted state.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• malformed input around byte {count}
• Encoded string reached maximum length: {utflen}
• Encoded string is too long: {utflen}
• malformed input: partial character at end
• Encoded string reached maximum length:

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/6f30fc55bdc8b34d. Report an issue: GitHub.