{"record":{"id":"ccb5b2e37605f3ab","repo":"apache/hadoop","slug":"incorrect-value-for-packet-payload-size-payload","errorCode":null,"errorMessage":"Incorrect value for packet payload size: ${payloadLen}","messagePattern":"Incorrect value for packet payload size: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketReceiver.java","lineNumber":167,"sourceCode":"      // The \"payload length\" includes its own length. Therefore it\n      // should never be less than 4 bytes\n      throw new IOException(\"Invalid payload length \" +\n          payloadLen);\n    }\n    int dataPlusChecksumLen = payloadLen - Ints.BYTES;\n    int headerLen = curPacketBuf.getShort();\n    if (headerLen < 0) {\n      throw new IOException(\"Invalid header length \" + headerLen);\n    }\n\n    LOG.trace(\"readNextPacket: dataPlusChecksumLen={}, headerLen={}\",\n        dataPlusChecksumLen, headerLen);\n\n    // Sanity check the buffer size so we don't allocate too much memory\n    // and OOME.\n    int totalLen = payloadLen + headerLen;\n    if (totalLen < 0 || totalLen > MAX_PACKET_SIZE) {\n      throw new IOException(\"Incorrect value for packet payload size: \" +\n                            payloadLen);\n    }\n\n    // Make sure we have space for the whole packet, and\n    // read it.\n    reallocPacketBuf(PacketHeader.PKT_LENGTHS_LEN +\n        dataPlusChecksumLen + headerLen);\n    curPacketBuf.clear();\n    curPacketBuf.position(PacketHeader.PKT_LENGTHS_LEN);\n    curPacketBuf.limit(PacketHeader.PKT_LENGTHS_LEN +\n        dataPlusChecksumLen + headerLen);\n    doReadFully(ch, in, curPacketBuf);\n    curPacketBuf.flip();\n    curPacketBuf.position(PacketHeader.PKT_LENGTHS_LEN);\n\n    // Extract the header from the front of the buffer (after the length prefixes)\n    byte[] headerBuf = new byte[headerLen];\n    curPacketBuf.get(headerBuf);","sourceCodeStart":149,"sourceCodeEnd":185,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketReceiver.java#L149-L185","documentation":"Sanity check in readNextPacket(): payloadLen + headerLen must not overflow to negative nor exceed MAX_PACKET_SIZE (from dfs.client.max-packet-size-ish key DFS_DATA_TRANSFER_MAX_PACKET_SIZE, default 16 MiB as HdfsClientConfigKeys default). Its purpose is to prevent OOME from a hostile/garbage length field causing a huge buffer allocation.","triggerScenarios":"A length field claiming a packet larger than 16MB: sender configured with oversized chunk/packet sizes, corrupt length bytes, or an attacker crafting headers. Also integer overflow when payloadLen and headerLen sum past Integer.MAX_VALUE.","commonSituations":"Custom/incompatible writer that emits jumbo packets, corrupted streams, security scanning against the data-transfer port, or exotic configs raising chunk sizes beyond the reader cap.","solutions":["If you deliberately increased chunk/packet sizes, raise dfs.datanode.transfer.max-packet-size consistently on BOTH ends within the same key/value.","Otherwise treat as corruption: retry on another replica and check node/network health.","Audit who can connect to data transfer ports if the pattern looks hostile."],"exampleFix":"// hdfs-site.xml — must match on writer and reader sides\n<property>\n  <name>dfs.datanode.transfer.max-packet-size</name>\n  <value>16777216</value> <!-- raise only if you also raised chunk sizes -->\n</property>","handlingStrategy":"validation","validationCode":"// Writer side: keep announced packet sizes within the reader cap before sending\nint totalLen = payloadLen + headerLen;\nif (totalLen < 0 || totalLen > 16 * 1024 * 1024) { // must match receiver's max-packet-size\n  throw new IllegalArgumentException(\"packet too large: \" + totalLen);\n}","typeGuard":null,"tryCatchPattern":"try { receiver.readNextPacket(); }\ncatch (IOException e) {\n  if (e.getMessage().contains(\"packet payload size\")) { /* abort stream; fix packet-size config mismatch */ }\n}","preventionTips":["If raising chunk/packet sizes by config, set dfs.datanode.transfer.max-packet-size identically on every node and client.","Never construct packets larger than the configured cap in custom DataNode/client code.","Treat unexpected occurrences as corruption: retry elsewhere and audit the host."],"tags":["hdfs","data-transfer","packet-size","corruption","memory-guard"],"backgroundTag":"protocol-stream-corruption","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}