apache/kafka · critical · InvalidRecordException

Invalid record size: expected {} bytes in record payload, bu

Error message

Invalid record size: expected {} bytes in record payload, but the record payload reached EOF.

What it means

DefaultRecord.readFrom(InputStream, ...) (line 289) reads a varint sizeOfBodyInBytes then attempts to read exactly that many bytes; if the stream returns fewer (EOF) before the body completes it throws InvalidRecordException. The declared record size does not match the bytes available, signalling truncation or corruption of the batch.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecord.java:289

        result = 31 * result + Long.hashCode(offset);
        result = 31 * result + Long.hashCode(timestamp);
        result = 31 * result + sequence;
        result = 31 * result + (key != null ? key.hashCode() : 0);
        result = 31 * result + (value != null ? value.hashCode() : 0);
        result = 31 * result + Arrays.hashCode(headers);
        return result;
    }

    public static DefaultRecord readFrom(InputStream input,
                                         long baseOffset,
                                         long baseTimestamp,
                                         int baseSequence,
                                         Long logAppendTime) throws IOException {
        int sizeOfBodyInBytes = ByteUtils.readVarint(input);
        ByteBuffer recordBuffer = ByteBuffer.allocate(sizeOfBodyInBytes);
        int bytesRead = Utils.readFully(input, recordBuffer);
        if (bytesRead != sizeOfBodyInBytes)
            throw new InvalidRecordException("Invalid record size: expected " + sizeOfBodyInBytes +
                " bytes in record payload, but the record payload reached EOF.");
        recordBuffer.flip(); // prepare for reading
        return readFrom(recordBuffer, sizeOfBodyInBytes, baseOffset, baseTimestamp,
                baseSequence, logAppendTime);
    }

    public static DefaultRecord readFrom(ByteBuffer buffer,
                                         long baseOffset,
                                         long baseTimestamp,
                                         int baseSequence,
                                         Long logAppendTime) {
        int sizeOfBodyInBytes = ByteUtils.readVarint(buffer);
        return readFrom(buffer, sizeOfBodyInBytes, baseOffset, baseTimestamp,
            baseSequence, logAppendTime);
    }

    private static DefaultRecord readFrom(ByteBuffer buffer,
                                          int sizeOfBodyInBytes,

View on GitHub (pinned to c31c9215e1)

Solutions

Inspect the segment with kafka-dump-log to confirm the truncation point.
Truncate or replace the bad segment, or restore from a healthy replica.
Check broker logs for unclean shutdown / I/O errors on the affected log dir.

Defensive patterns

Strategy: try-catch

Validate before calling

// If you control the InputStream, verify available bytes match the declared body size before readFrom
import java.io.InputStream;
import org.apache.kafka.common.utils.ByteUtils;

// read the varint body size yourself, then ensure the stream can deliver it
int sizeOfBodyInBytes = ByteUtils.readVarint(input);
// best-effort: available() is a hint, not a guarantee
if (input.available() >= sizeOfBodyInBytes || /* or you know exact remaining */ false) {
    // proceed; otherwise the read will hit EOF
}

Type guard

import java.nio.ByteBuffer;

static boolean canReadRecord(ByteBuffer buf) {
    if (buf == null || buf.remaining() < 1) return false;
    ByteBuffer dup = buf.duplicate();
    try {
        int size = ByteUtils.readVarint(dup);
        return dup.remaining() >= size;
    } catch (Exception e) {
        return false;
    }
}

// usage (ByteBuffer overload): if (canReadRecord(buf)) { DefaultRecord.readFrom(buf, ...); }

Try / catch

try {
    DefaultRecord.readFrom(input, baseOffset, baseTimestamp, baseSequence, logAppendTime);
} catch (org.apache.kafka.common.InvalidRecordException e) {
    // declared body size exceeded available bytes: truncation/corruption;
    // stop reading this batch, do not retry the same truncated stream
} catch (java.io.IOException e) {
    // underlying stream I/O failure; handle separately
}

Prevention

A size/EOF mismatch means the batch is truncated or corrupted; it is not transient and must not be retried on the same bytes.
When reading from a network stream, prefer the ByteBuffer overload after fully reading the batch so the size check is exact.
Validate batch-level lengths before descending into individual records so corruption is caught at the right boundary.
Quarantine the offending segment/offset and log base offset + declared size for forensics.
Do not conflate this with a transient IOException; only the latter warrants retry.

When it happens

Trigger: Reading a record batch from a socket, file, or ByteBuffer-backed stream where the batch is cut off mid-record; encountered on consumer fetch, replica fetch, or log scan paths that use the InputStream overload of readFrom.

Common situations: Corrupted/truncated log segment on disk; network interruption during fetch producing a short batch; partial write from a crashed broker; mismatched batch-length header.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/24c5bfebca2c8a90.json. Report an issue: GitHub.