apache/iceberg · error · UTFDataFormatException

malformed input around byte {count}

Error message

malformed input around byte {count}

What it means

During modified UTF-8 decoding in readLongUTF, a 2-byte sequence's second byte must be a continuation byte (10xxxxxx, i.e. & 0xC0 == 0x80). If it is not, the byte sequence is not valid modified UTF-8 and the decoder throws at the offending byte position.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java:138

        case 3:
        case 4:
        case 5:
        case 6:
        case 7:
          /* 0xxxxxxx */
          count++;
          chararr[chararrCount++] = (char) ch;
          break;
        case 12:
        case 13:
          /* 110x xxxx 10xx xxxx */
          count += 2;
          if (count > utflen) {
            throw new UTFDataFormatException("malformed input: partial character at end");
          }
          char2 = bytearr[count - 1];
          if ((char2 & 0xC0) != 0x80) {
            throw new UTFDataFormatException("malformed input around byte " + count);
          }
          chararr[chararrCount++] = (char) (((ch & 0x1F) << 6) | (char2 & 0x3F));
          break;
        case 14:
          /* 1110 xxxx 10xx xxxx 10xx xxxx */
          count += 3;
          if (count > utflen) {
            throw new UTFDataFormatException("malformed input: partial character at end");
          }
          char2 = bytearr[count - 2];
          char3 = bytearr[count - 1];
          if (((char2 & 0xC0) != 0x80) || ((char3 & 0xC0) != 0x80)) {
            throw new UTFDataFormatException("malformed input around byte " + (count - 1));
          }
          chararr[chararrCount++] =
              (char) (((ch & 0x0F) << 12) | ((char2 & 0x3F) << 6) | (char3 & 0x3F));
          break;
        default:

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Ensure both writer and reader use SerializerHelper.writeLongUTF/readLongUTF symmetrically
2. Validate/clean input data to be valid UTF-8 before serializing
3. Use binary (byte[]) types instead of strings for non-text data
4. Recover data from a non-corrupted checkpoint

try {
  return SerializerHelper.readLongUTF(in);
} catch (UTFDataFormatException e) {
  LOG.error("malformed UTF data at {}", e.getMessage(), e);
  throw new IOException("invalid serialized string data", e);
}

When it happens

Trigger: Reading a stream containing bytes that break the UTF-8 continuation-byte rule — e.g. random binary data interpreted as UTF, or strings written by an incompatible encoder.

Common situations: Deserializing data produced by a different serializer version; feeding binary blobs into string fields; corrupted state files.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• malformed input around byte
• Encoded string reached maximum length: {utflen}
• Encoded string is too long: {utflen}
• malformed input: partial character at end
• Encoded string reached maximum length:

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/caa98ec0d565b5b0. Report an issue: GitHub.