{"record":{"id":"193d1ee9031d9b56","repo":"apache/hadoop","slug":"invalid-packet-data-length-in-packet-header-excee","errorCode":null,"errorMessage":"Invalid packet: data length in packet header exceeds data length received. dataPlusChecksumLen=${dataPlusChecksumLen} header: ${header}","messagePattern":"Invalid packet: data length in packet header exceeds data length received\\. dataPlusChecksumLen=(.+?) header: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketReceiver.java","lineNumber":194,"sourceCode":"    curPacketBuf.position(PacketHeader.PKT_LENGTHS_LEN);\n    curPacketBuf.limit(PacketHeader.PKT_LENGTHS_LEN +\n        dataPlusChecksumLen + headerLen);\n    doReadFully(ch, in, curPacketBuf);\n    curPacketBuf.flip();\n    curPacketBuf.position(PacketHeader.PKT_LENGTHS_LEN);\n\n    // Extract the header from the front of the buffer (after the length prefixes)\n    byte[] headerBuf = new byte[headerLen];\n    curPacketBuf.get(headerBuf);\n    if (curHeader == null) {\n      curHeader = new PacketHeader();\n    }\n    curHeader.setFieldsFromData(payloadLen, headerBuf);\n\n    // Compute the sub-slices of the packet\n    int checksumLen = dataPlusChecksumLen - curHeader.getDataLen();\n    if (checksumLen < 0) {\n      throw new IOException(\"Invalid packet: data length in packet header \" +\n          \"exceeds data length received. dataPlusChecksumLen=\" +\n          dataPlusChecksumLen + \" header: \" + curHeader);\n    }\n\n    reslicePacket(headerLen, checksumLen, curHeader.getDataLen());\n  }\n\n  /**\n   * Rewrite the last-read packet on the wire to the given output stream.\n   */\n  public void mirrorPacketTo(DataOutputStream mirrorOut) throws IOException {\n    Preconditions.checkState(!useDirectBuffers,\n        \"Currently only supported for non-direct buffers\");\n    mirrorOut.write(curPacketBuf.array(),\n        curPacketBuf.arrayOffset(),\n        curPacketBuf.remaining());\n  }\n","sourceCodeStart":176,"sourceCodeEnd":212,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketReceiver.java#L176-L212","documentation":"After parsing the packet header, PacketReceiver computes checksumLen = dataPlusChecksumLen - header.getDataLen(); a negative result means the header claims more data bytes than the packet's payload length can hold — an internally inconsistent header, i.e. corruption or desynchronization. The message dumps both the payload length and full header for diagnosis.","triggerScenarios":"Truncated or corrupted packets where the data length field in PacketHeader exceeds the announced payload: failing disks producing short reads on the sender, TCP stream desync after an earlier framing error, or incompatible PacketHeader layouts between versions.","commonSituations":"DataNode disks going bad (short/garbage writes into the pipeline), rolling-upgrade version mixes, transport-level corruption, flaky direct-buffer handling in custom builds.","solutions":["Retry the operation so a different replica/pipeline is used; DFSClient-level retries usually mask transient cases.","Run 'hdfs fsck -blockId' on the reported block and check sender DataNode logs for disk errors.","Ensure uniform Hadoop versions across the pipeline.","If persistent, capture traffic and compare announced lengths vs actual bytes to locate the corrupting hop."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  receiver.readNextPacket();\n} catch (IOException e) {\n  if (e.getMessage().contains(\"data length in packet header exceeds\")) {\n    // inconsistent header vs payload — drop the replica/reader and retry\n    throw new StreamCorruptException(e.getMessage(), e);\n  }\n  throw e;\n}","preventionTips":["Monitor sender-side DataNode disks; short/garbled writes are the common producer of inconsistent headers.","Retry reads to land on a different replica; report repeated offenders via bad-file reporting.","Keep pipeline versions uniform during rolling upgrades."],"tags":["hdfs","data-transfer","packet","corruption","checksum"],"backgroundTag":"protocol-stream-corruption","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}