{"id":"24c5bfebca2c8a90","repo":"apache/kafka","slug":"invalid-record-size-expected-bytes-in-record-p","errorCode":null,"errorMessage":"Invalid record size: expected {} bytes in record payload, but the record payload reached EOF.","messagePattern":"Invalid record size: expected (.+?) bytes in record payload, but the record payload reached EOF\\.","errorType":"exception","errorClass":"InvalidRecordException","httpStatus":null,"severity":"critical","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecord.java","lineNumber":289,"sourceCode":"        result = 31 * result + Long.hashCode(offset);\n        result = 31 * result + Long.hashCode(timestamp);\n        result = 31 * result + sequence;\n        result = 31 * result + (key != null ? key.hashCode() : 0);\n        result = 31 * result + (value != null ? value.hashCode() : 0);\n        result = 31 * result + Arrays.hashCode(headers);\n        return result;\n    }\n\n    public static DefaultRecord readFrom(InputStream input,\n                                         long baseOffset,\n                                         long baseTimestamp,\n                                         int baseSequence,\n                                         Long logAppendTime) throws IOException {\n        int sizeOfBodyInBytes = ByteUtils.readVarint(input);\n        ByteBuffer recordBuffer = ByteBuffer.allocate(sizeOfBodyInBytes);\n        int bytesRead = Utils.readFully(input, recordBuffer);\n        if (bytesRead != sizeOfBodyInBytes)\n            throw new InvalidRecordException(\"Invalid record size: expected \" + sizeOfBodyInBytes +\n                \" bytes in record payload, but the record payload reached EOF.\");\n        recordBuffer.flip(); // prepare for reading\n        return readFrom(recordBuffer, sizeOfBodyInBytes, baseOffset, baseTimestamp,\n                baseSequence, logAppendTime);\n    }\n\n    public static DefaultRecord readFrom(ByteBuffer buffer,\n                                         long baseOffset,\n                                         long baseTimestamp,\n                                         int baseSequence,\n                                         Long logAppendTime) {\n        int sizeOfBodyInBytes = ByteUtils.readVarint(buffer);\n        return readFrom(buffer, sizeOfBodyInBytes, baseOffset, baseTimestamp,\n            baseSequence, logAppendTime);\n    }\n\n    private static DefaultRecord readFrom(ByteBuffer buffer,\n                                          int sizeOfBodyInBytes,","sourceCodeStart":271,"sourceCodeEnd":307,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecord.java#L271-L307","documentation":"DefaultRecord.readFrom(InputStream, ...) (line 289) reads a varint sizeOfBodyInBytes then attempts to read exactly that many bytes; if the stream returns fewer (EOF) before the body completes it throws InvalidRecordException. The declared record size does not match the bytes available, signalling truncation or corruption of the batch.","triggerScenarios":"Reading a record batch from a socket, file, or ByteBuffer-backed stream where the batch is cut off mid-record; encountered on consumer fetch, replica fetch, or log scan paths that use the InputStream overload of readFrom.","commonSituations":"Corrupted/truncated log segment on disk; network interruption during fetch producing a short batch; partial write from a crashed broker; mismatched batch-length header.","solutions":["Inspect the segment with kafka-dump-log to confirm the truncation point.","Truncate or replace the bad segment, or restore from a healthy replica.","Check broker logs for unclean shutdown / I/O errors on the affected log dir."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// If you control the InputStream, verify available bytes match the declared body size before readFrom\nimport java.io.InputStream;\nimport org.apache.kafka.common.utils.ByteUtils;\n\n// read the varint body size yourself, then ensure the stream can deliver it\nint sizeOfBodyInBytes = ByteUtils.readVarint(input);\n// best-effort: available() is a hint, not a guarantee\nif (input.available() >= sizeOfBodyInBytes || /* or you know exact remaining */ false) {\n    // proceed; otherwise the read will hit EOF\n}","typeGuard":"import java.nio.ByteBuffer;\n\nstatic boolean canReadRecord(ByteBuffer buf) {\n    if (buf == null || buf.remaining() < 1) return false;\n    ByteBuffer dup = buf.duplicate();\n    try {\n        int size = ByteUtils.readVarint(dup);\n        return dup.remaining() >= size;\n    } catch (Exception e) {\n        return false;\n    }\n}\n\n// usage (ByteBuffer overload): if (canReadRecord(buf)) { DefaultRecord.readFrom(buf, ...); }","tryCatchPattern":"try {\n    DefaultRecord.readFrom(input, baseOffset, baseTimestamp, baseSequence, logAppendTime);\n} catch (org.apache.kafka.common.InvalidRecordException e) {\n    // declared body size exceeded available bytes: truncation/corruption;\n    // stop reading this batch, do not retry the same truncated stream\n} catch (java.io.IOException e) {\n    // underlying stream I/O failure; handle separately\n}","preventionTips":["A size/EOF mismatch means the batch is truncated or corrupted; it is not transient and must not be retried on the same bytes.","When reading from a network stream, prefer the ByteBuffer overload after fully reading the batch so the size check is exact.","Validate batch-level lengths before descending into individual records so corruption is caught at the right boundary.","Quarantine the offending segment/offset and log base offset + declared size for forensics.","Do not conflate this with a transient IOException; only the latter warrants retry."],"tags":["record-format","data-corruption","deserialization"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}