{"record":{"id":"00fe16af60de6909","repo":"apache/hadoop","slug":"error-deserializing-buffer","errorCode":null,"errorMessage":"Error deserializing buffer.","messagePattern":"Error deserializing buffer\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-streaming/src/main/java/org/apache/hadoop/record/Utils.java","lineNumber":237,"sourceCode":"   * @return\n   */\n  static String toCSVBuffer(Buffer buf) {\n    StringBuilder sb = new StringBuilder(\"#\");\n    sb.append(buf.toString());\n    return sb.toString();\n  }\n  \n  /**\n   * Converts a CSV-serialized representation of buffer to a new\n   * Buffer\n   * @param s CSV-serialized representation of buffer\n   * @throws java.io.IOException\n   * @return Deserialized Buffer\n   */\n  static Buffer fromCSVBuffer(String s)\n    throws IOException {\n    if (s.charAt(0) != '#') {\n      throw new IOException(\"Error deserializing buffer.\");\n    }\n    if (s.length() == 1) { return new Buffer(); }\n    int blen = (s.length()-1)/2;\n    byte[] barr = new byte[blen];\n    for (int idx = 0; idx < blen; idx++) {\n      char c1 = s.charAt(2*idx+1);\n      char c2 = s.charAt(2*idx+2);\n      barr[idx] = (byte)Integer.parseInt(\"\"+c1+c2, 16);\n    }\n    return new Buffer(barr);\n  }\n  \n  private static int utf8LenForCodePoint(final int cpt) throws IOException {\n    if (cpt >=0 && cpt <= 0x7F) {\n      return 1;\n    }\n    if (cpt >= 0x80 && cpt <= 0x07FF) {\n      return 2;","sourceCodeStart":219,"sourceCodeEnd":255,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-streaming/src/main/java/org/apache/hadoop/record/Utils.java#L219-L255","documentation":"Utils.fromCSVBuffer() expects the CSV wire form of a Buffer to start with '#' followed by pairs of hex digits (as written by toCSVBuffer). If the first character is not '#', it throws IOException 'Error deserializing buffer.' — the value being read is not a CSV-serialized buffer at all.","triggerScenarios":"CsvRecordInput.readBuffer() (or direct fromCSVBuffer calls) receiving a plain string, a hex blob without the '#' sentinel, or a value shifted from another field/column of the record.","commonSituations":"Schema drift between writer and reader: the writer wrote a string where the reader expects a buffer, records truncated in transit, or hand-built CSV payloads missing the '#' prefix.","solutions":["Confirm writer and reader field order/types match — the '#' sentinel is written only by toCSVBuffer()","Log and hex-dump the failing field to identify what was actually supplied","Regenerate the serialized data with the matching org.apache.hadoop.record version","Replace record/CSV serialization with Avro, which has an explicit schema"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"static boolean isCsvEncodedBuffer(String s) {\n  if (s == null || s.isEmpty() || s.charAt(0) != '#') return false;\n  String hex = s.substring(1);\n  return hex.isEmpty() || hex.matches(\"([0-9a-fA-F]{2})+\");\n}","typeGuard":null,"tryCatchPattern":"catch IOException from readBuffer/fromCSVBuffer; report field position and raw value, treat as schema mismatch (string vs buffer).","preventionTips":["Keep writer and reader record schemas in lockstep — a '#' sentinel means the writer used toCSVBuffer","Verify field order with a small round-trip test whenever the record layout changes","Prefer Avro with an explicit schema for any new interchange"],"tags":["hadoop-record","csv","deserialization","buffer","deprecated"],"backgroundTag":"malformed-input-data","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}