{"record":{"id":"4821c92dd2ea3b58","repo":"apache/hadoop","slug":"invalid-payload-length-payloadlen","errorCode":null,"errorMessage":"Invalid payload length ${payloadLen}","messagePattern":"Invalid payload length (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketReceiver.java","lineNumber":151,"sourceCode":"    // HLEN:      Header length\n    //            = length(HEADER)\n    //\n    // HEADER:    the actual packet header fields, encoded in protobuf\n    // CHECKSUMS: the crcs for the data chunk. May be missing if\n    //            checksums were not requested\n    // DATA       the actual block data\n    Preconditions.checkState(curHeader == null || !curHeader.isLastPacketInBlock());\n\n    curPacketBuf.clear();\n    curPacketBuf.limit(PacketHeader.PKT_LENGTHS_LEN);\n    doReadFully(ch, in, curPacketBuf);\n    curPacketBuf.flip();\n    int payloadLen = curPacketBuf.getInt();\n\n    if (payloadLen < Ints.BYTES) {\n      // The \"payload length\" includes its own length. Therefore it\n      // should never be less than 4 bytes\n      throw new IOException(\"Invalid payload length \" +\n          payloadLen);\n    }\n    int dataPlusChecksumLen = payloadLen - Ints.BYTES;\n    int headerLen = curPacketBuf.getShort();\n    if (headerLen < 0) {\n      throw new IOException(\"Invalid header length \" + headerLen);\n    }\n\n    LOG.trace(\"readNextPacket: dataPlusChecksumLen={}, headerLen={}\",\n        dataPlusChecksumLen, headerLen);\n\n    // Sanity check the buffer size so we don't allocate too much memory\n    // and OOME.\n    int totalLen = payloadLen + headerLen;\n    if (totalLen < 0 || totalLen > MAX_PACKET_SIZE) {\n      throw new IOException(\"Incorrect value for packet payload size: \" +\n                            payloadLen);\n    }","sourceCodeStart":133,"sourceCodeEnd":169,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketReceiver.java#L133-L169","documentation":"PacketReceiver.readNextPacket() reads the 4-byte payload length prefix of a data-transfer packet; values below 4 bytes (Ints.BYTES) are impossible because the payload length includes its own 4-byte field, so the stream is corrupt or misframed. Thrown as IOException on the reading side (client BlockReader or DataNode mirror receiver).","triggerScenarios":"A misaligned read on the data-transfer socket: earlier bytes consumed incorrectly, a peer speaking a different protocol version, memory/disk corruption producing garbage length fields, or a truncated first packet after a protocol error reply instead of data.","commonSituations":"Version mismatch between client and DataNode packet formats, TCP streams corrupted by faulty hardware or middleboxes, DataNode returning an out-of-band error before the pipeline stalled, or downstream mirror DataNodes with mismatched versions during rolling upgrades.","solutions":["Verify all pipeline nodes and the client run compatible Hadoop versions (rolling-upgrade compatibility rules).","Check DataNode logs around the failure; retry the read/write — DFSClient will choose another replica.","Inspect network hardware/middleboxes on the data path if corruption recurs cluster-wide.","Capture the socket (tcpdump) to confirm where framing diverges if the issue persists."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  receiver.readNextPacket();\n} catch (IOException e) {\n  if (e.getMessage().contains(\"Invalid payload length\")) {\n    // framing corrupt — abort this stream; upper layer retries on another replica\n    throw new StreamCorruptException(e.getMessage(), e);\n  }\n  throw e;\n}","preventionTips":["Pin client and DataNode versions during rolling upgrades to compatible ranges.","Watch DataNode disk/network error rates; corrupt replicas and flaky NICs are the usual sources.","Retry reads at the application level so another replica is used when a stream desyncs."],"tags":["hdfs","data-transfer","corruption","wire-protocol","packet"],"backgroundTag":"protocol-stream-corruption","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}