apache/iceberg · error · UTFDataFormatException

malformed input around byte

Error message

malformed input around byte 

What it means

During modified-UTF-8 decoding, a continuation byte must have the form 10xxxxxx. When the byte following a 2-byte or 3-byte lead byte does not match that pattern, readLongUTF throws UTFDataFormatException 'malformed input around byte N' naming the byte offset. The data is not valid modified UTF-8.

Source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java:138

        case 3:
        case 4:
        case 5:
        case 6:
        case 7:
          /* 0xxxxxxx */
          count++;
          chararr[chararrCount++] = (char) ch;
          break;
        case 12:
        case 13:
          /* 110x xxxx 10xx xxxx */
          count += 2;
          if (count > utflen) {
            throw new UTFDataFormatException("malformed input: partial character at end");
          }
          char2 = bytearr[count - 1];
          if ((char2 & 0xC0) != 0x80) {
            throw new UTFDataFormatException("malformed input around byte " + count);
          }
          chararr[chararrCount++] = (char) (((ch & 0x1F) << 6) | (char2 & 0x3F));
          break;
        case 14:
          /* 1110 xxxx 10xx xxxx 10xx xxxx */
          count += 3;
          if (count > utflen) {
            throw new UTFDataFormatException("malformed input: partial character at end");
          }
          char2 = bytearr[count - 2];
          char3 = bytearr[count - 1];
          if (((char2 & 0xC0) != 0x80) || ((char3 & 0xC0) != 0x80)) {
            throw new UTFDataFormatException("malformed input around byte " + (count - 1));
          }
          chararr[chararrCount++] =
              (char) (((ch & 0x0F) << 12) | ((char2 & 0x3F) << 6) | (char3 & 0x3F));
          break;
        default:

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Ensure the same helper performed the write (writeLongUTF paired with readLongUTF).
2. Validate the byte offset/stream position — the deserialization may be misaligned with the buffer.
3. Regenerate the payload; if corruption persists, check storage/network integrity and checkpoint/serializer compatibility.

Example fix

// before
in.readUTF(); // writer used DataOutputStream.writeUTF
String s = SerializerHelper.readLongUTF(...);
// after
helper.writeLongUTF(out, s); // pair writes and reads on the same helper
String s = helper.readLongUTF(in);

// java
for (int i = 0; i < bytes.length; ) {
  int b = bytes[i] & 0xFF;
  int n = b < 0x80 ? 1 : (b < 0xE0 ? 2 : (b < 0xF0 ? 3 : -1));
  if (n < 0 || i + n > bytes.length) { throw new IOException("not valid modified UTF-8 at " + i); }
  for (int j = 1; j < n; j++) { if ((bytes[i + j] & 0xC0) != 0x80) { throw new IOException("bad continuation at " + (i + j)); } }
  i += n;
}

boolean looksLikeModifiedUtf8(byte[] bytes) {
  for (int i = 0; i < bytes.length; i++) {
    int b = bytes[i] & 0xFF;
    if (b >= 0x80 && b < 0xC0 && (i == 0 || (bytes[i - 1] & 0xC0) == 0x80)) { return false; }
  }
  return true;
}

try {
  String s = helper.readLongUTF(in);
} catch (UTFDataFormatException e) {
  if (e.getMessage().startsWith("malformed input around byte")) {
    // check writer/reader pairing and stream alignment
  } else { throw e; }
}

When it happens

Trigger: Feeding bytes not produced by writeLongUTF (e.g. standard UTF-8 with embedded NULs encoded differently, or arbitrary binary) into readLongUTF; bit corruption in transit.

Common situations: Mixing DataOutput.writeUTF and SerializerHelper.readLongUTF or vice versa; deserializing from the wrong offset in a shared buffer; corrupted checkpoints.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• malformed input: partial character at end
• malformed input: partial character at end
• malformed input: partial character at end
• Encoded string reached maximum length: {utflen}
• Encoded string is too long: {utflen}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/3c73d70600af4133. Report an issue: GitHub.