apache/kafka · error · CorruptRecordException

Record for partition ${partition.topicPartition()} at offset

Error message

Record for partition ${partition.topicPartition()} at offset ${record.offset()} is invalid, cause: ${e.getMessage()}

What it means

CorruptRecordException re-thrown from ShareCompletedFetch.maybeEnsureValid when an individual Record (not the whole batch) fails record.ensureValid() under CRC checking. The message embeds the topic-partition and the record's own offset, letting you pinpoint the single offending record inside an otherwise-valid batch. It exists separately from the batch-level check because a batch header can be valid while an inner record is not.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareCompletedFetch.java:431

    }

    private void maybeEnsureValid(RecordBatch batch, boolean checkCrcs) {
        if (checkCrcs && batch.magic() >= RecordBatch.MAGIC_VALUE_V2) {
            try {
                batch.ensureValid();
            } catch (CorruptRecordException e) {
                throw new CorruptRecordException("Record batch for partition " + partition.topicPartition()
                        + " at offset " + batch.baseOffset() + " is invalid, cause: " + e.getMessage());
            }
        }
    }

    private void maybeEnsureValid(final Record record, final boolean checkCrcs) {
        if (checkCrcs) {
            try {
                record.ensureValid();
            } catch (CorruptRecordException e) {
                throw new CorruptRecordException("Record for partition " + partition.topicPartition()
                        + " at offset " + record.offset() + " is invalid, cause: " + e.getMessage());
            }
        }
    }

    private void maybeCloseRecordStream() {
        if (records != null) {
            records.close();
            records = null;
        }
    }

    private static class OffsetAndDeliveryCount {
        final long offset;
        final short deliveryCount;

        OffsetAndDeliveryCount(long offset, short deliveryCount) {
            this.offset = offset;

View on GitHub (pinned to c31c9215e1)

Solutions

Retry the poll to rule out a transient transport-level bit flip.
Investigate the producer that wrote the specific offset; reproduce with kafka-console-consumer --property check.crcs=true against the same partition.
If the record is unrecoverable and blocking progress, skip it via the share consumer acknowledgement flow (acknowledge as RELEASE/ABANDON per your retry policy) and quarantine the offset for replay/reprocessing.
Audit producer serializers/compressors and broker disk for the affected segment.

Defensive patterns

Strategy: try-catch

Validate before calling

// No pre-call validation: per-record CRC failure surfaces only during deserialization/iteration.
// If you never want this exception, disable CRC checking (not recommended):
props.put("check.crcs", "false");

Try / catch

// Wrap record iteration so a single bad record does not poison the whole batch.
import org.apache.kafka.common.errors.CorruptRecordException;

for (ConsumerRecord<K,V> r : records) {
    try {
        process(r);
    } catch (CorruptRecordException e) {
        log.warn("Skipping corrupt record {}-{}: {}",
                 r.topic(), r.offset(), e.getMessage());
        shareConsumer.acknowledge(...); // advance past the bad offset
    }
}

Prevention

Catch at the per-record scope, not the whole poll() — one corrupt record should not abort the batch.
Always acknowledge past a corrupt offset in share mode, otherwise the group re-fetches the same bad record forever.
Export corrupt-record counters to your alerting stack; spikes correlate with broker disk errors or producer serialization bugs.
Keep check.crcs=true (default); the CPU cost is trivial versus silent data loss.

When it happens

Trigger: Polling a share consumer with check.crcs=true where the batch passes maybeEnsureValid but a subsequent record.ensureValid() throws CorruptRecordException. Happens while streaming records out of the batch iterator in ShareCompletedFetch.

Common situations: Same family as batch-level corruption but localized to one record: partial write that was fsync'd mid-record, decompression of a compacted/garbage-collected record that left an inconsistent size/offset, or an upstream producer that hit a serialization bug mid-batch. Also seen with custom interceptors or faulty compression codecs on the producer side.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/86577a5959232fec.json. Report an issue: GitHub.