apache/kafka · critical · IOException

Block checksum mismatch

Error message

Block checksum mismatch

What it means

Thrown by readBlock() in Lz4BlockInputStream when the FLG block-checksum bit was set by the producer and the XXHash32 recomputed over the (decompressed-then-stored) block bytes does not equal the 4-byte checksum stored immediately after the block. It signals silent data corruption of one LZ4 block — the bytes received are not the bytes that were compressed.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockInputStream.java:203

                final int bufferSize = DECOMPRESSOR.decompress(in, in.position(), blockSize, decompressionBuffer, 0,
                    maxBlockSize);
                decompressionBuffer.position(0);
                decompressionBuffer.limit(bufferSize);
                decompressedBuffer = decompressionBuffer;
            } catch (LZ4Exception e) {
                throw new IOException(e);
            }
        } else {
            decompressedBuffer = in.slice();
            decompressedBuffer.limit(blockSize);
        }

        // verify checksum
        if (flg.isBlockChecksumSet()) {
            int hash = CHECKSUM.hash(in, in.position(), blockSize, 0);
            in.position(in.position() + blockSize);
            if (hash != in.getInt()) {
                throw new IOException(BLOCK_HASH_MISMATCH);
            }
        } else {
            in.position(in.position() + blockSize);
        }
    }

    @Override
    public int read() throws IOException {
        if (finished) {
            return -1;
        }
        if (available() == 0) {
            readBlock();
        }
        if (finished) {
            return -1;
        }

View on GitHub (pinned to c31c9215e1)

Solutions

1. Treat as data corruption: re-fetch the partition from another replica / from an earlier offset to get an uncorrupted copy.
2. Inspect broker disk health, dmesg and NIC/SRAM error counters; run memtest on the affected host.
3. Verify no non-Kafka producer or transforming proxy (MirrorMaker 2 with re-encoding, custom Connect SMT) is altering compressed payloads without recomputing the LZ4 block checksum.
4. If reproducing locally, dump the failing block bytes and compare against the producer's source to localize corruption to producer, broker, or network leg.

Example fix

// producer side: never strip or rewrite compressed bytes after compression;
// always let Kafka's CompressionType.LZ4 build the frame so checksums match.
// (No code fix applies on the consumer side — the block IS corrupt; re-fetch.)

// Kafka record batches carry their own CRC32C; verify it BEFORE decompression
// so a checksum mismatch is caught at the batch layer, not inside LZ4.
import org.apache.kafka.common.record.DefaultRecordBatch;
if (batch instanceof DefaultRecordBatch && !((DefaultRecordBatch) batch).isValidCrc()) {
    throw new IOException("Refusing to decompress batch: outer CRC32C check failed");
}

null

try {
    try (Lz4BlockInputStream in = new Lz4BlockInputStream(payload, BufferSupplier.NO_CACHING, true)) {
        // read decompressed bytes
    }
} catch (IOException e) {
    if (e.getMessage() != null && e.getMessage().contains(Lz4BlockInputStream.BLOCK_HASH_MISMATCH)) {
        // Block-level XXHash32 failed: data is corrupt in transit or at rest.
        log.error("LZ4 block checksum mismatch on {}, purging from cache and refetching", tp);
        consumer.seek(tp, offset); // refetch this batch
        return;
    }
    throw e;
}

When it happens

Trigger: Decompressing a block whose FLG.isBlockChecksumSet() == true (Kafka always sets this when writing LZ4), where CHECKSUM.hash(in, position, blockSize, 0) != in.getInt(). Reached only when flg.blockChecksum==1, on every block read in readBlock() line 199-203.

Common situations: Disk or memory bit-rot on the broker; faulty NIC / RAM causing silent corruption without TCP detection; a third-party producer that wrote a checksum over the wrong bytes (e.g. over compressed vs uncompressed data); JDK or native-bridge issues; intermediary (proxy, mirror-maker) that re-encodes the payload but copies the old checksum. Rare in healthy Kafka deployments because Kafka always writes block checksums, so this is a real corruption signal, not a benign warning.

Related errors

• Stream frame descriptor corrupted
• mark not supported
• reset not supported
• Kafka has detected a buggy lz4-java library (< 1.4.x) on the
• The stream is already closed

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/199ba7ba17444355.json. Report an issue: GitHub.