{"record":{"id":"806ae00f51d6a6b0","repo":"apache/hadoop","slug":"invalid-header-length-headerlen","errorCode":null,"errorMessage":"Invalid header length ${headerLen}","messagePattern":"Invalid header length (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketReceiver.java","lineNumber":157,"sourceCode":"    // DATA       the actual block data\n    Preconditions.checkState(curHeader == null || !curHeader.isLastPacketInBlock());\n\n    curPacketBuf.clear();\n    curPacketBuf.limit(PacketHeader.PKT_LENGTHS_LEN);\n    doReadFully(ch, in, curPacketBuf);\n    curPacketBuf.flip();\n    int payloadLen = curPacketBuf.getInt();\n\n    if (payloadLen < Ints.BYTES) {\n      // The \"payload length\" includes its own length. Therefore it\n      // should never be less than 4 bytes\n      throw new IOException(\"Invalid payload length \" +\n          payloadLen);\n    }\n    int dataPlusChecksumLen = payloadLen - Ints.BYTES;\n    int headerLen = curPacketBuf.getShort();\n    if (headerLen < 0) {\n      throw new IOException(\"Invalid header length \" + headerLen);\n    }\n\n    LOG.trace(\"readNextPacket: dataPlusChecksumLen={}, headerLen={}\",\n        dataPlusChecksumLen, headerLen);\n\n    // Sanity check the buffer size so we don't allocate too much memory\n    // and OOME.\n    int totalLen = payloadLen + headerLen;\n    if (totalLen < 0 || totalLen > MAX_PACKET_SIZE) {\n      throw new IOException(\"Incorrect value for packet payload size: \" +\n                            payloadLen);\n    }\n\n    // Make sure we have space for the whole packet, and\n    // read it.\n    reallocPacketBuf(PacketHeader.PKT_LENGTHS_LEN +\n        dataPlusChecksumLen + headerLen);\n    curPacketBuf.clear();","sourceCodeStart":139,"sourceCodeEnd":175,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketReceiver.java#L139-L175","documentation":"After reading the payload length, PacketReceiver reads a 2-byte header length; since it is read into a short-based field, any value with the high bit set decodes negative and is rejected. A negative header length cannot be valid, so the packet framing is corrupt.","triggerScenarios":"Corrupt or desynchronized data-transfer stream where the 2 bytes after the payload length are garbage (high bit set); same root causes as invalid payload length — version mismatch, misframed reads, transport corruption.","commonSituations":"Rolling upgrades mixing packet-header layouts, NIC/TCP offload corruption, reading from a mirror DataNode that aborted mid-stream, fuzzed or intercepted connections.","solutions":["Retry the block read/write to land on a different replica/node.","Align Hadoop versions across the write pipeline (client + all DataNodes).","Check DataNode and client logs for the first protocol error that led to desync; fix that root cause.","If recurring on specific nodes, suspect hardware and run diagnostics on that host."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  receiver.readNextPacket();\n} catch (IOException e) {\n  if (e.getMessage().contains(\"Invalid header length\")) {\n    // stream misframed/corrupt — close reader, retry on another replica\n  } else { throw e; }\n}","preventionTips":["Align versions across write pipelines; mixed packet-header layouts desync streams.","Investigate the first protocol error in the logs — later 'invalid length' errors are downstream symptoms.","Replace/schedule diagnostics for hosts that repeatedly produce framing errors."],"tags":["hdfs","data-transfer","corruption","wire-protocol","packet-header"],"backgroundTag":"protocol-stream-corruption","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}