{"record":{"id":"3c73d70600af4133","repo":"apache/iceberg","slug":"malformed-input-around-byte-3c73d7","errorCode":null,"errorMessage":"malformed input around byte ","messagePattern":"malformed input around byte ","errorType":"exception","errorClass":"UTFDataFormatException","httpStatus":null,"severity":"error","filePath":"flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java","lineNumber":138,"sourceCode":"        case 3:\n        case 4:\n        case 5:\n        case 6:\n        case 7:\n          /* 0xxxxxxx */\n          count++;\n          chararr[chararrCount++] = (char) ch;\n          break;\n        case 12:\n        case 13:\n          /* 110x xxxx 10xx xxxx */\n          count += 2;\n          if (count > utflen) {\n            throw new UTFDataFormatException(\"malformed input: partial character at end\");\n          }\n          char2 = bytearr[count - 1];\n          if ((char2 & 0xC0) != 0x80) {\n            throw new UTFDataFormatException(\"malformed input around byte \" + count);\n          }\n          chararr[chararrCount++] = (char) (((ch & 0x1F) << 6) | (char2 & 0x3F));\n          break;\n        case 14:\n          /* 1110 xxxx 10xx xxxx 10xx xxxx */\n          count += 3;\n          if (count > utflen) {\n            throw new UTFDataFormatException(\"malformed input: partial character at end\");\n          }\n          char2 = bytearr[count - 2];\n          char3 = bytearr[count - 1];\n          if (((char2 & 0xC0) != 0x80) || ((char3 & 0xC0) != 0x80)) {\n            throw new UTFDataFormatException(\"malformed input around byte \" + (count - 1));\n          }\n          chararr[chararrCount++] =\n              (char) (((ch & 0x0F) << 12) | ((char2 & 0x3F) << 6) | (char3 & 0x3F));\n          break;\n        default:","sourceCodeStart":120,"sourceCodeEnd":156,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/util/SerializerHelper.java#L120-L156","documentation":"During modified-UTF-8 decoding, a continuation byte must have the form 10xxxxxx. When the byte following a 2-byte or 3-byte lead byte does not match that pattern, readLongUTF throws UTFDataFormatException 'malformed input around byte N' naming the byte offset. The data is not valid modified UTF-8.","triggerScenarios":"Feeding bytes not produced by writeLongUTF (e.g. standard UTF-8 with embedded NULs encoded differently, or arbitrary binary) into readLongUTF; bit corruption in transit.","commonSituations":"Mixing DataOutput.writeUTF and SerializerHelper.readLongUTF or vice versa; deserializing from the wrong offset in a shared buffer; corrupted checkpoints.","solutions":["Ensure the same helper performed the write (writeLongUTF paired with readLongUTF).","Validate the byte offset/stream position — the deserialization may be misaligned with the buffer.","Regenerate the payload; if corruption persists, check storage/network integrity and checkpoint/serializer compatibility."],"exampleFix":"// before\nin.readUTF(); // writer used DataOutputStream.writeUTF\nString s = SerializerHelper.readLongUTF(...);\n// after\nhelper.writeLongUTF(out, s); // pair writes and reads on the same helper\nString s = helper.readLongUTF(in);","handlingStrategy":"try-catch","validationCode":"// java\nfor (int i = 0; i < bytes.length; ) {\n  int b = bytes[i] & 0xFF;\n  int n = b < 0x80 ? 1 : (b < 0xE0 ? 2 : (b < 0xF0 ? 3 : -1));\n  if (n < 0 || i + n > bytes.length) { throw new IOException(\"not valid modified UTF-8 at \" + i); }\n  for (int j = 1; j < n; j++) { if ((bytes[i + j] & 0xC0) != 0x80) { throw new IOException(\"bad continuation at \" + (i + j)); } }\n  i += n;\n}","typeGuard":"boolean looksLikeModifiedUtf8(byte[] bytes) {\n  for (int i = 0; i < bytes.length; i++) {\n    int b = bytes[i] & 0xFF;\n    if (b >= 0x80 && b < 0xC0 && (i == 0 || (bytes[i - 1] & 0xC0) == 0x80)) { return false; }\n  }\n  return true;\n}","tryCatchPattern":"try {\n  String s = helper.readLongUTF(in);\n} catch (UTFDataFormatException e) {\n  if (e.getMessage().startsWith(\"malformed input around byte\")) {\n    // check writer/reader pairing and stream alignment\n  } else { throw e; }\n}","preventionTips":["Use the same SerializerHelper for write and read.","Ensure stream offsets align (read exactly what was written).","Detect corruption with checksums on serialized payloads.","Keep serializer versions consistent across job restarts/checkpoints."],"tags":["flink","serialization","utf","corruption"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}