apache/kafka · critical · CorruptRecordException

Record is corrupt (stored crc = {}, computed crc = {})

Error message

Record is corrupt (stored crc = {}, computed crc = {})

What it means

Thrown by DefaultRecordBatch.ensureValid when the batch's stored CRC32C (in the header) does not equal the freshly computed CRC32C over the batch body. The v2 batch carries a CRC the producer computes; any byte divergence between what was produced and what is now in the buffer makes ensureValid reject it. CorruptRecordException.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecordBatch.java:157

    private final ByteBuffer buffer;

    DefaultRecordBatch(ByteBuffer buffer) {
        this.buffer = buffer;
    }

    @Override
    public byte magic() {
        return buffer.get(MAGIC_OFFSET);
    }

    @Override
    public void ensureValid() {
        if (sizeInBytes() < RECORD_BATCH_OVERHEAD)
            throw new CorruptRecordException("Record batch is corrupt (the size " + sizeInBytes() +
                    " is smaller than the minimum allowed overhead " + RECORD_BATCH_OVERHEAD + ")");

        if (!isValid())
            throw new CorruptRecordException("Record is corrupt (stored crc = " + checksum()
                    + ", computed crc = " + computeChecksum() + ")");
    }

    /**
     * Gets the base timestamp of the batch which is used to calculate the record timestamps from the deltas.
     *
     * @return The base timestamp
     */
    public long baseTimestamp() {
        return buffer.getLong(BASE_TIMESTAMP_OFFSET);
    }

    @Override
    public long maxTimestamp() {
        return buffer.getLong(MAX_TIMESTAMP_OFFSET);
    }

    @Override

View on GitHub (pinned to c31c9215e1)

Solutions

1. Confirm whether corruption is isolated to one batch/segment with kafka-dump-log --deep-iteration; if so, drop/replay that segment.
2. Run hardware diagnostics (memtest, smartctl long test, fsck) — recurring CRC mismatches across topics strongly indicate failing RAM or disk.
3. Ensure no code path mutates the batch buffer after MemoryRecordsBuilder closes it (the CRC is finalized at close); check custom interceptors/serializers and zero-copy paths.
4. Align producer/broker/client compression library versions (e.g. lz4-java, snappy, zstd-jni) so decompressed bytes match what was compressed at produce time.

Example fix

// before: mutating the batch buffer after the builder has computed the CRC
builder.close();
batchBuf.putInt(BASE_OFFSET_OFFSET, newOffset); // CRC now stale

// after: rebuild via the builder so CRC is recomputed after all fields are set
MemoryRecordsBuilder b = MemoryRecords.builder(buf, magic, compression,
    timestampType, baseOffset);
b.append(...); // b.close() finalizes CRC once all fields are correct

// CRC mismatch is detected during ensureValid() on a buffer already on the wire/disk.
// There is nothing to validate before the call — the CRC is recomputed internally.
// The only preventive measure is to ensure your producer side is the one writing CRCs
// (never bypass KafkaProducer / MemoryRecords builder).

// Distinguish CRC corruption (skip) from transient fetch failures (retry):
try {
    for (RecordBatch b : records) b.ensureValid();
} catch (CorruptRecordException e) { // CRC mismatch
    log.error("CRC failure on {} @ {} — likely bit-rot or tampering", partition, offset, e);
    alert(CRC_FAILURE, partition, offset);
    consumer.seek(partition, offset + 1);
}

When it happens

Trigger: Produced on the broker append path (Log.append / validateMessagesAndAssignOffsets), in consumer/fetch validation, and during log recovery when the bytes under the CRC field have been altered or partially overwritten. Typical: silent disk corruption, RAM bit-flips, a non-Kafka writer modifying batch bytes, or a JVM/library bug double-wrapping/altering the buffer after CRC computation.

Common situations: Faulty disk or failing DIMM causing bit rot on a cold segment; memcpy/zero-copy bug in a custom producer that rewrites a field after the CRC was set; compression library version skew where a recompressed body no longer matches the producer's CRC; partial-page writes after a power loss.

Related errors

• Record batch is corrupt (the size {} is smaller than the min
• Incorrect declared batch size, premature EOF reached
• Found invalid record count {} in magic v{} batch
• Encountered corrupt message when fetching offset {} for topi
• Timestamp type must be provided to compute attributes for me

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/6ad106f4aac5e434.json. Report an issue: GitHub.