What causes "Reserved bits must be 0" in apache/kafka?

Confirm the data was produced by a Kafka client (which always emits reserved=0); if not, re-encode with Kafka's CompressionType.LZ4.

apache/kafka · error · RuntimeException

Reserved bits must be 0

Error message

Reserved bits must be 0

What it means

Thrown by FLG.validate() (invoked from the FLG constructor / FLG.fromByte) when the two low-order reserved bits of the LZ4 Frame Descriptor FLG byte are non-zero. Per the LZ4 frame spec these bits MUST be zero for forward compatibility; a non-zero value means the frame was produced by an unknown/extension format that this implementation refuses to interpret.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockOutputStream.java:315

            int blockIndependence = (flg >>> 5) & 1;
            int version = (flg >>> 6) & 3;

            return new FLG(reserved,
                           contentChecksum,
                           contentSize,
                           blockChecksum,
                           blockIndependence,
                           version);
        }

        public byte toByte() {
            return (byte) (((reserved & 3) << 0) | ((contentChecksum & 1) << 2)
                    | ((contentSize & 1) << 3) | ((blockChecksum & 1) << 4) | ((blockIndependence & 1) << 5) | ((version & 3) << 6));
        }

        private void validate() {
            if (reserved != 0) {
                throw new RuntimeException("Reserved bits must be 0");
            }
            if (blockIndependence != 1) {
                throw new RuntimeException("Dependent block stream is unsupported");
            }
            if (version != VERSION) {
                throw new RuntimeException(String.format("Version %d is unsupported", version));
            }
        }

        public boolean isContentChecksumSet() {
            return contentChecksum == 1;
        }

        public boolean isContentSizeSet() {
            return contentSize == 1;
        }

        public boolean isBlockChecksumSet() {

View on GitHub (pinned to c31c9215e1)

Solutions

Confirm the data was produced by a Kafka client (which always emits reserved=0); if not, re-encode with Kafka's CompressionType.LZ4.
Inspect the raw FLG byte at the failing offset and compare against the LZ4 frame spec to identify which extension set the reserved bits.
If the producer is yours, ensure FLG is constructed only via the public FLG(boolean blockChecksum) constructor (reserved defaults to 0).
Re-fetch from another replica to rule out in-flight corruption of the FLG byte.

Example fix

// before: hand-built FLG with a custom reserved bit
FLG flg = new FLG(1, 0, 0, 1, 1, 1); // reserved=1 -> throws

// after: use the public constructor, reserved stays 0
FLG flg = new FLG(true); // blockChecksum=true, all reserved bits 0

Defensive patterns

Strategy: try-catch

Validate before calling

// FLG reserved bits are bits 0-1 of the FLG byte; they must be zero.
// Pre-validate a raw LZ4 frame before constructing FLG.fromByte().
byte flgByte = frameBuf.get(flgOffset);
if ((flgByte & 0x03) != 0) {
    throw new IOException("Refusing malformed LZ4 frame: FLG reserved bits set (0x" + Integer.toHexString(flgByte & 0xff) + ")");
}

Type guard

null

Try / catch

try {
    // FLG.fromByte is reached indirectly through new Lz4BlockInputStream(...)
    try (Lz4BlockInputStream in = new Lz4BlockInputStream(payload, BufferSupplier.NO_CACHING, false)) {
        // consume
    }
} catch (RuntimeException e) {
    if (e.getMessage() != null && e.getMessage().contains("Reserved bits must be 0")) {
        // Malformed LZ4 frame header — quarantine the record and continue.
        log.warn("Discarding malformed LZ4 frame (reserved bits set) on {}", tp);
        return;
    }
    throw e;
}

Prevention

This path is only reachable when decompressing externally-supplied LZ4 frames (i.e. corrupted records or data not produced by Kafka); never attempt to construct FLG by hand.
When ingesting LZ4 data from non-Kafka sources, validate the frame header (magic + FLG + BD + HC) before passing to Kafka's decompressor.
Keep producer and consumer Kafka client versions aligned so FLG bytes are always produced by validated code.
Log the raw FLG byte on failure to speed root-cause of silent corruption.

When it happens

Trigger: FLG.fromByte(b) parsing a Frame Descriptor whose bits 0-1 are set, e.g. during Lz4BlockInputStream.readHeader() at line 127 while decoding a frame produced by a non-Kafka LZ4 implementation (or by a future/extended LZ4 format). Also reachable if a producer manually constructs FLG with reserved != 0.

Common situations: Consuming LZ4-compressed Kafka records produced by a third-party client that sets reserved FLG bits; bit-rot in the FLG byte; future LZ4 frame version that defines new flag bits in the reserved range; testing with hand-crafted LZ4 frames; reading Kafka data through a tool that rewrites headers.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/8fd84772c9afbe5d.json. Report an issue: GitHub.