{"record":{"id":"922c26779f660a3a","repo":"apache/hadoop","slug":"malformed-varint","errorCode":null,"errorMessage":"Malformed varint","messagePattern":"Malformed varint","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ProtoUtil.java","lineNumber":73,"sourceCode":"    } else {\n      result |= (tmp & 0x7f) << 7;\n      if ((tmp = in.readByte()) >= 0) {\n        result |= tmp << 14;\n      } else {\n        result |= (tmp & 0x7f) << 14;\n        if ((tmp = in.readByte()) >= 0) {\n          result |= tmp << 21;\n        } else {\n          result |= (tmp & 0x7f) << 21;\n          result |= (tmp = in.readByte()) << 28;\n          if (tmp < 0) {\n            // Discard upper 32 bits.\n            for (int i = 0; i < 5; i++) {\n              if (in.readByte() >= 0) {\n                return result;\n              }\n            }\n            throw new IOException(\"Malformed varint\");\n          }\n        }\n      }\n    }\n    return result;\n  }\n\n  \n  /** \n   * This method creates the connection context  using exactly the same logic\n   * as the old connection context as was done for writable where\n   * the effective and real users are set based on the auth method.\n   *\n   * @param protocol protocol.\n   * @param ugi ugi.\n   * @param authMethod authMethod.\n   * @return IpcConnectionContextProto.\n   */","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ProtoUtil.java#L55-L91","documentation":"ProtoUtil.readRawVarint32() decodes a protobuf base-128 varint from a DataInput, as used in Hadoop RPC handshakes and length-prefixed protobuf frames. A 32-bit varint needs at most 5 bytes, and up to 5 further continuation bytes of a 64-bit encoding are discarded; if after those the stream still has the continuation bit set (byte < 0), the encoding is invalid and it throws IOException(\"Malformed varint\").","triggerScenarios":"Feeding readRawVarint32 a stream that is not positioned at a varint: RPC frame desync after an earlier field was misread; client and server Hadoop versions disagreeing on wire serialization; bytes corrupted or mangled by SASL/QOP or a middlebox; custom code reusing the helper on arbitrary buffers.","commonSituations":"Mixed Hadoop client/server versions on one cluster; hand-rolled RPC clients writing lengths as fixed ints or strings; tests writing raw payloads where a varint length is expected; proxies altering the byte stream.","solutions":["Verify both ends run compatible Hadoop/RPC protocol versions","Instrument the writer, not just the reader — confirm every length is written with the matching varint encoding (protobuf CodedOutputStream / DataOutputByteBuffer paths)","On this IOException, drop the connection/session; do not attempt to resynchronize mid-stream","For custom protocols, write with writeRawVarint32-style encoding and pair it with this reader"],"exampleFix":"// before: writer emits a fixed 4-byte length, reader expects varint\ndout.writeInt(len);\n\n// after: matching varint encoding on both sides\n// writer\norg.apache.hadoop.ipc.protobuf.RpcHeaderProtosHelper.writeRawVarint32(dout, len);\n// reader\nint len = ProtoUtil.readRawVarint32(din);","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { len = ProtoUtil.readRawVarint32(in); } catch (IOException e) { // \"Malformed varint\" => stream is desynced; never resync mid-stream\n  throw new IOException(\"RPC stream corrupted (malformed varint); closing connection\", e); }","preventionTips":["Pin client and server to compatible Hadoop/RPC protocol versions","Always write lengths with the varint encoding that matches this reader","On malformed varint, close the connection rather than continuing to parse","Add frame-level logging when hand-rolling RPC clients"],"tags":["hadoop","java","rpc","protobuf","varint","wire-protocol"],"backgroundTag":"varint-decode-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}