{"id":"199ba7ba17444355","repo":"apache/kafka","slug":"block-checksum-mismatch","errorCode":null,"errorMessage":"Block checksum mismatch","messagePattern":"Block checksum mismatch","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockInputStream.java","lineNumber":203,"sourceCode":"                final int bufferSize = DECOMPRESSOR.decompress(in, in.position(), blockSize, decompressionBuffer, 0,\n                    maxBlockSize);\n                decompressionBuffer.position(0);\n                decompressionBuffer.limit(bufferSize);\n                decompressedBuffer = decompressionBuffer;\n            } catch (LZ4Exception e) {\n                throw new IOException(e);\n            }\n        } else {\n            decompressedBuffer = in.slice();\n            decompressedBuffer.limit(blockSize);\n        }\n\n        // verify checksum\n        if (flg.isBlockChecksumSet()) {\n            int hash = CHECKSUM.hash(in, in.position(), blockSize, 0);\n            in.position(in.position() + blockSize);\n            if (hash != in.getInt()) {\n                throw new IOException(BLOCK_HASH_MISMATCH);\n            }\n        } else {\n            in.position(in.position() + blockSize);\n        }\n    }\n\n    @Override\n    public int read() throws IOException {\n        if (finished) {\n            return -1;\n        }\n        if (available() == 0) {\n            readBlock();\n        }\n        if (finished) {\n            return -1;\n        }\n","sourceCodeStart":185,"sourceCodeEnd":221,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockInputStream.java#L185-L221","documentation":"Thrown by readBlock() in Lz4BlockInputStream when the FLG block-checksum bit was set by the producer and the XXHash32 recomputed over the (decompressed-then-stored) block bytes does not equal the 4-byte checksum stored immediately after the block. It signals silent data corruption of one LZ4 block — the bytes received are not the bytes that were compressed.","triggerScenarios":"Decompressing a block whose FLG.isBlockChecksumSet() == true (Kafka always sets this when writing LZ4), where CHECKSUM.hash(in, position, blockSize, 0) != in.getInt(). Reached only when flg.blockChecksum==1, on every block read in readBlock() line 199-203.","commonSituations":"Disk or memory bit-rot on the broker; faulty NIC / RAM causing silent corruption without TCP detection; a third-party producer that wrote a checksum over the wrong bytes (e.g. over compressed vs uncompressed data); JDK or native-bridge issues; intermediary (proxy, mirror-maker) that re-encodes the payload but copies the old checksum. Rare in healthy Kafka deployments because Kafka always writes block checksums, so this is a real corruption signal, not a benign warning.","solutions":["Treat as data corruption: re-fetch the partition from another replica / from an earlier offset to get an uncorrupted copy.","Inspect broker disk health, dmesg and NIC/SRAM error counters; run memtest on the affected host.","Verify no non-Kafka producer or transforming proxy (MirrorMaker 2 with re-encoding, custom Connect SMT) is altering compressed payloads without recomputing the LZ4 block checksum.","If reproducing locally, dump the failing block bytes and compare against the producer's source to localize corruption to producer, broker, or network leg."],"exampleFix":"// producer side: never strip or rewrite compressed bytes after compression;\n// always let Kafka's CompressionType.LZ4 build the frame so checksums match.\n// (No code fix applies on the consumer side — the block IS corrupt; re-fetch.)","handlingStrategy":"try-catch","validationCode":"// Kafka record batches carry their own CRC32C; verify it BEFORE decompression\n// so a checksum mismatch is caught at the batch layer, not inside LZ4.\nimport org.apache.kafka.common.record.DefaultRecordBatch;\nif (batch instanceof DefaultRecordBatch && !((DefaultRecordBatch) batch).isValidCrc()) {\n    throw new IOException(\"Refusing to decompress batch: outer CRC32C check failed\");\n}","typeGuard":"null","tryCatchPattern":"try {\n    try (Lz4BlockInputStream in = new Lz4BlockInputStream(payload, BufferSupplier.NO_CACHING, true)) {\n        // read decompressed bytes\n    }\n} catch (IOException e) {\n    if (e.getMessage() != null && e.getMessage().contains(Lz4BlockInputStream.BLOCK_HASH_MISMATCH)) {\n        // Block-level XXHash32 failed: data is corrupt in transit or at rest.\n        log.error(\"LZ4 block checksum mismatch on {}, purging from cache and refetching\", tp);\n        consumer.seek(tp, offset); // refetch this batch\n        return;\n    }\n    throw e;\n}","preventionTips":["Always validate the outer record-batch CRC32C (RecordBatch.isValidCrc) before decompression; it catches bit-flips earlier and more cheaply than the LZ4 block hash.","If you broker or proxy records, keep block checksums enabled (do not hand-craft FLG bytes with blockChecksum=0).","Treat a block checksum mismatch as a signal of storage/network corruption — refetch from an in-sync replica rather than retrying blindly.","Do not reuse or mutate ByteBuffer contents returned by the consumer; they may still be referenced by the decompressor."],"tags":["compression","lz4","checksum","data-corruption","kafka-clients"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}