apache/kafka · error · CorruptRecordException

Record is corrupt (crc could not be retrieved as the record

Error message

Record is corrupt (crc could not be retrieved as the record is too small, size = {})

What it means

Thrown by LegacyRecord.ensureValid when the record buffer is smaller than RECORD_OVERHEAD_V0 (14 bytes), so there is not even room for the CRC field. This means the on-disk record is truncated or corrupted below the minimum v0 header size, and the CRC cannot be read at all. It is a CorruptRecordException — Kafka treats the record as unparseable.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/record/internal/LegacyRecord.java:132

     * Retrieve the previously computed CRC for this record
     */
    public long checksum() {
        return ByteUtils.readUnsignedInt(buffer, CRC_OFFSET);
    }

    /**
     * Returns true if the crc stored with the record matches the crc computed off the record contents
     */
    public boolean isValid() {
        return sizeInBytes() >= RECORD_OVERHEAD_V0 && checksum() == computeChecksum();
    }

    /**
     * Throw an CorruptRecordException if isValid is false for this record
     */
    public void ensureValid() {
        if (sizeInBytes() < RECORD_OVERHEAD_V0)
            throw new CorruptRecordException("Record is corrupt (crc could not be retrieved as the record is too "
                    + "small, size = " + sizeInBytes() + ")");

        if (!isValid())
            throw new CorruptRecordException("Record is corrupt (stored crc = " + checksum()
                    + ", computed crc = " + computeChecksum() + ")");
    }

    /**
     * The complete serialized size of this record in bytes (including crc, header attributes, etc), but
     * excluding the log overhead (offset and record size).
     * @return the size in bytes
     */
    public int sizeInBytes() {
        return buffer.limit();
    }

    /**
     * The length of the key in bytes

View on GitHub (pinned to c31c9215e1)

Solutions

Run the broker's log recovery (delete the recovery-point checkpoint so the segment is re-validated); the corrupt tail record will be truncated.
If the corruption is on a non-active segment, restore the segment from an in-sync replica rather than repairing by hand.
Upgrade message.format.version to v2 (2.x) so newer batches carry length prefixes that make truncation detection cleaner.
Enable unclean.leader.election only when you accept potential data loss; otherwise the ISR-based truncation avoids the partial write in the first place.

Example fix

// before: validating a record from a possibly-truncated segment
LegacyRecord record = new LegacyRecord(buffer);
record.ensureValid();  // throws CorruptRecordException if buffer < 14 bytes

// after: skip undersized buffers during recovery and log them
LegacyRecord record = new LegacyRecord(buffer);
if (buffer.remaining() >= LegacyRecord.RECORD_OVERHEAD_V0) {
    record.ensureValid();
} else {
    log.warn("Skipping undersized legacy record of {} bytes during recovery", buffer.remaining());
}

Defensive patterns

Strategy: try-catch

Validate before calling

if (record.sizeInBytes() < LegacyRecord.RECORD_OVERHEAD_V0) {
    // undersized / truncated frame; skip rather than call ensureValid()
    log.warn("Skipping record: {} bytes is smaller than v0 overhead {}",
             record.sizeInBytes(), LegacyRecord.RECORD_OVERHEAD_V0);
    continue;
}

Try / catch

try {
    record.ensureValid();
} catch (CorruptRecordException e) {
    // record frame is too small to even carry a CRC; quarantine and continue
    log.warn("Discarding undersized corrupt record", e);
    metrics.corruptRecord();
}

Prevention

Prefer message format v2 (magic=2) over legacy v0/v1; it has stronger framing.
Gate on record.sizeInBytes() >= LegacyRecord.RECORD_OVERHEAD_V0 before touching fields.
Handle CorruptRecordException in consumers and skip-and-commit rather than blocking the partition.

When it happens

Trigger: Invoking LegacyRecord.ensureValid() on a record whose buffer.limit() < 14. Reached via the v0/v1 record iterator when reading legacy message-format segments, during log validation/recovery, or in clients consuming old-format records from a broker still on message.format.version 0 or 1.

Common situations: Reading a log segment truncated mid-record (power loss, disk full, unclean shutdown), partial writes from a crashed producer/broker, or a segment file whose last batch is incomplete. Common when message.format.version is still set to 0/1 on an old cluster being upgraded.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/c9b1fe7e8a799558.json. Report an issue: GitHub.